Last week I upped my ChatGPT prompting game.
I’ve got a fair bit of experience with ChatGPT, but I’ve never spent much time investigating the intricacies of so-called prompt engineering, so I was interested in the course from DeepLearning.AI which was developed in collaboration with engineers from OpenAI.
In the course, I learned some new things
1. Delimiters are critical.
You can use triple quotes, backticks or dashes, and HTML code-like tags or even just angle brackets (I’m sure square brackets would work too) to isolate your input from the prompt itself. Delimiters are not only important for making your prompt clearer but they are critical in development to avoid user prompt injections (think SQL injection hacks).
For example: “Summarise the text delimited by triple backticks into a single sentence.
2. You should ask for structured output (e.g. JSON or XML)
You can prompt the model to give you structured output, which will enable you to write code to process the responses. You can specify the format or let the model come up with its own standard – but beware that you use a low “temperature” to make sure the responses are predictable (this is only a setting for the API, on the web client it’s set quite high so it’s more creative).
For example: “Generate a list of three made-up book titles along with their authors and genres.
Provide them in JSON format with the following keys: book_id, title, author, genre.”
3. Use conditions to limit hallucinations
Ask the model to check if certain conditions are satisfied and to let you know if they aren’t. I found this helped to avoid hallucinations.
For example, this prompt: “Please give me a quote about writing from Ali Smith” resulted in ChatGPT either making up a quote or taking a quote from another author. But “Please give me a quote about writing from Ali Smith. If you don’t know of any quotes by Ali Smith, please say `I can’t find any notable quotes attributed to Ali Smith`.” resulted in the ChatGPT saying it couldn’t find any quotes.
4. Give the model “time to think”
This was an interesting concept that they covered in some depth. The basic concept is that if the task is complex you will need to instruct the model to follow an algorithm to complete the task instead of just jumping in.
For example if you give ChatGPT a problem and an answer to check – ask it to answer the problem itself and then compare its answer to the answer you provide.
Try adding the phrase “step by step” to your prompt.
There is no perfect prompt – you need to have a process for developing a prompt for your scenario, and iterate until it works consistently.
The course also shows you how to achieve common tasks with LLMs like ChatGPT and gives you an opportunity to experiment yourself using their Jupyter notebook.