Last week I wrote about a bunch of new ChatCraft features we shipped, including the ability to edit any message in a chat. This week we explored what this new edit feature could enable, extending the idea to the system prompt, which was previously hidden and fixed. By combining a visible system prompt with an edit button, we've been able to unlock some really interesting abilities in the app.
A lot has been written about the need to create detailed prompts for LLMs. However, I've been equally interested in exploring what I could do by altering the behaviour of the model via a good system prompt. System prompts (or system messages) are often the first message in a chat. Most apps don't expose them to users. From the OpenAI docs:
Typically, a conversation is formatted with a system message first, followed by alternating user and assistant messages. The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. However note that the system message is optional and the model’s behavior without a system message is likely to be similar to using a generic message such as "You are a helpful assistant."
The system prompt helps to set the tone of your interaction with a model. Do you want output in a particular format? Are certain types of responses required and/or forbidden? Are there underlying assumptions that the model needs to know about when interacting with the user? The system prompt, like stage direction in a play, lets you guide the model on how to best play the part of a character in the story that's about to unfold.
As I mentioned above, most apps try to hide their system prompt, and there's lots of interest when someone can goad a model into revealing what it is. I understand why many apps can't disclose this, and don't want people trying to circumvent it. However, one of the things I love about ChatCraft is that we don't have to establish a wall around how the system works. Instead, everything is editable!
Our current system prompt looks like this:
I am ChatCraft, a web-based, expert programming AI assistant. I help programmers learn, experiment, and be more creative with code.
I follow these rules when responding:
- Use GitHub flavored Markdown
- ALWAYS include the programming language name (js) or type of data (csv) at the start of Markdown code blocks
- Format ALL lines of code to 80 characters or fewer
- Use Mermaid diagrams when discussing visual topicsIt's worked quite well, not least because we aren't trying to dramatically alter the nature of the LLMs we use (it can be hard to get a model to go against its training. For example, try getting a model to respond about code, but never give actual source code).
As we've built ChatCraft, we've iterated on this prompt a few times to fix bugs, or nudge the responses toward or away from certain types of output. It's pretty easy to change, but requires you to submit a PR to GitHub.
With our new edit feature, we wanted to see if we could make this process more open to experiments by users. For example, I've wanted to test-drive the ChatCraft UI as a way to do research into biology and ecology vs. programming, and Taras has been exploring business topics. Obviously these changes aren't something that we're going to land in git, so doing it was possible but frustrating.
By replacing our old "I'm a helpful assistant!" greeting message with an actual system prompt we expose in the UI, we make it possible to let the user take a more active role in establishing the limits and potential of the conversation. Here's the new UI:
The system prompt is truncated, but clicking "More..." will reveal it. The user can also click "Edit" in the message's menu to customize it:
Now it's easy to do both minor tweaks or major surgery. Here are 2 examples of the same prompt, but each chat uses a customized system prompt. Notice how different each conversation becomes as a result of these changes–one in French, the other in short sentences:
These examples are really minor adjustments, but they show how powerful the technique can be. I think that as we begin to include the system prompt in the scope of what we allow the user to alter, we'll discover new ways to work with the tool more quickly.
Taras and I are excited to apply this technique to our upcoming explorations into adding functions to ChatCraft. Imagine being able to use ChatCraft to help you write functional tools that can then be used to solve problems in the conversation. We think it's possible, and this is taking us one step closer!
Let us know if you manage to do something interesting with a custom system prompt or have more ideas.