From 1c16c58a7b6775ee00b7de7fec13efeacb95d14f Mon Sep 17 00:00:00 2001 From: Elvis Saravia Date: Wed, 8 Feb 2023 21:42:23 +0000 Subject: [PATCH] add more examples --- guides/prompts-advanced-usage.md | 80 +++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 1 deletion(-) diff --git a/guides/prompts-advanced-usage.md b/guides/prompts-advanced-usage.md index 873871f..99e6383 100644 --- a/guides/prompts-advanced-usage.md +++ b/guides/prompts-advanced-usage.md @@ -128,4 +128,82 @@ Then you bought 5 more apples, so now you had 11 apples. Finally, you ate 1 apple, so you would remain with 10 apples. ``` -It's actually impressive that this simple prompt is effective at this task. This is particularly useful where you don't too many examples to use in the prompt. \ No newline at end of file +It's actually impressive that this simple prompt is effective at this task. This is particularly useful where you don't have too many examples to use in the prompt. + +--- + +## Self-Consistency + +Perhaps one of the more advanced techniques out there for prompt engineering is self-consistency. Proposed by [Wang et al. (2022)](https://arxiv.org/pdf/2203.11171.pdf), self-consistency aims "to replace the naive greedy decoding used in chain-of-thought prompting". The idea is to sample multiple, diverse reasoning paths through few-shot CoT, and use the generations to select the most consistent answer. This helps to boost the performance of CoT prompting ont asks involving arithmetic and commonsense reasoning. + +Let's try the following example for arithmetic reasoning: + +``` +When I was 6 my sister was half my age. Now +I’m 70 how old is my sister? +``` + +Output +``` +35 +``` + +The output is obviously wrong! How may we improve this with self-consistency? Let's try it out. We will use the few-shot exemplars from Wang et al. 2022 (Table 17): + +``` +Q: There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done, +there will be 21 trees. How many trees did the grove workers plant today? +A: We start with 15 trees. Later we have 21 trees. The difference must be the number of trees they planted. +So, they must have planted 21 - 15 = 6 trees. The answer is 6. + +Q: If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot? +A: There are 3 cars in the parking lot already. 2 more arrive. Now there are 3 + 2 = 5 cars. The answer is 5. + +Q: Leah had 32 chocolates and her sister had 42. If they ate 35, how many pieces do they have left in total? +A: Leah had 32 chocolates and Leah’s sister had 42. That means there were originally 32 + 42 = 74 +chocolates. 35 have been eaten. So in total they still have 74 - 35 = 39 chocolates. The answer is 39. + +Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops +did Jason give to Denny? +A: Jason had 20 lollipops. Since he only has 12 now, he must have given the rest to Denny. The number of +lollipops he has given to Denny must have been 20 - 12 = 8 lollipops. The answer is 8. + +Q: Shawn has five toys. For Christmas, he got two toys each from his mom and dad. How many toys does +he have now? +A: He has 5 toys. He got 2 from mom, so after that he has 5 + 2 = 7 toys. Then he got 2 more from dad, so +in total he has 7 + 2 = 9 toys. The answer is 9. + +Q: There were nine computers in the server room. Five more computers were installed each day, from +monday to thursday. How many computers are now in the server room? +A: There are 4 days from monday to thursday. 5 computers were added each day. That means in total 4 * 5 = +20 computers were added. There were 9 computers in the beginning, so now there are 9 + 20 = 29 computers. +The answer is 29. + +Q: Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, he lost 2 more. How many +golf balls did he have at the end of wednesday? +A: Michael initially had 58 balls. He lost 23 on Tuesday, so after that he has 58 - 23 = 35 balls. On +Wednesday he lost 2 more so now he has 35 - 2 = 33 balls. The answer is 33. + +Q: Olivia has $23. She bought five bagels for $3 each. How much money does she have left? +A: She bought 5 bagels for $3 each. This means she spent 5 + +Q: When I was 6 my sister was half my age. Now I’m 70 how old is my sister? +A: +``` + +Output 1: +``` +When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70 - 3 = 67. The answer is 67. +``` + +Output 2: +``` +When the narrator was 6, his sister was half his age, which is 3. Now that the narrator is 70, his sister would be 70 - 3 = 67 years old. The answer is 67. +``` + +Output 3: +``` +When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70/2 = 35. The answer is 35. +``` + +Computing for the final answer involves a few steps (check out the paper for the details) but for the sake of simplicity, we can clearly see that there is already a majority answer emerging so that would essentially become the final answer. \ No newline at end of file