add more examples
parent
73e26fc914
commit
1c16c58a7b
|
@ -128,4 +128,82 @@ Then you bought 5 more apples, so now you had 11 apples.
|
|||
Finally, you ate 1 apple, so you would remain with 10 apples.
|
||||
```
|
||||
|
||||
It's actually impressive that this simple prompt is effective at this task. This is particularly useful where you don't too many examples to use in the prompt.
|
||||
It's actually impressive that this simple prompt is effective at this task. This is particularly useful where you don't have too many examples to use in the prompt.
|
||||
|
||||
---
|
||||
|
||||
## Self-Consistency
|
||||
|
||||
Perhaps one of the more advanced techniques out there for prompt engineering is self-consistency. Proposed by [Wang et al. (2022)](https://arxiv.org/pdf/2203.11171.pdf), self-consistency aims "to replace the naive greedy decoding used in chain-of-thought prompting". The idea is to sample multiple, diverse reasoning paths through few-shot CoT, and use the generations to select the most consistent answer. This helps to boost the performance of CoT prompting ont asks involving arithmetic and commonsense reasoning.
|
||||
|
||||
Let's try the following example for arithmetic reasoning:
|
||||
|
||||
```
|
||||
When I was 6 my sister was half my age. Now
|
||||
I’m 70 how old is my sister?
|
||||
```
|
||||
|
||||
Output
|
||||
```
|
||||
35
|
||||
```
|
||||
|
||||
The output is obviously wrong! How may we improve this with self-consistency? Let's try it out. We will use the few-shot exemplars from Wang et al. 2022 (Table 17):
|
||||
|
||||
```
|
||||
Q: There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done,
|
||||
there will be 21 trees. How many trees did the grove workers plant today?
|
||||
A: We start with 15 trees. Later we have 21 trees. The difference must be the number of trees they planted.
|
||||
So, they must have planted 21 - 15 = 6 trees. The answer is 6.
|
||||
|
||||
Q: If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot?
|
||||
A: There are 3 cars in the parking lot already. 2 more arrive. Now there are 3 + 2 = 5 cars. The answer is 5.
|
||||
|
||||
Q: Leah had 32 chocolates and her sister had 42. If they ate 35, how many pieces do they have left in total?
|
||||
A: Leah had 32 chocolates and Leah’s sister had 42. That means there were originally 32 + 42 = 74
|
||||
chocolates. 35 have been eaten. So in total they still have 74 - 35 = 39 chocolates. The answer is 39.
|
||||
|
||||
Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops
|
||||
did Jason give to Denny?
|
||||
A: Jason had 20 lollipops. Since he only has 12 now, he must have given the rest to Denny. The number of
|
||||
lollipops he has given to Denny must have been 20 - 12 = 8 lollipops. The answer is 8.
|
||||
|
||||
Q: Shawn has five toys. For Christmas, he got two toys each from his mom and dad. How many toys does
|
||||
he have now?
|
||||
A: He has 5 toys. He got 2 from mom, so after that he has 5 + 2 = 7 toys. Then he got 2 more from dad, so
|
||||
in total he has 7 + 2 = 9 toys. The answer is 9.
|
||||
|
||||
Q: There were nine computers in the server room. Five more computers were installed each day, from
|
||||
monday to thursday. How many computers are now in the server room?
|
||||
A: There are 4 days from monday to thursday. 5 computers were added each day. That means in total 4 * 5 =
|
||||
20 computers were added. There were 9 computers in the beginning, so now there are 9 + 20 = 29 computers.
|
||||
The answer is 29.
|
||||
|
||||
Q: Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, he lost 2 more. How many
|
||||
golf balls did he have at the end of wednesday?
|
||||
A: Michael initially had 58 balls. He lost 23 on Tuesday, so after that he has 58 - 23 = 35 balls. On
|
||||
Wednesday he lost 2 more so now he has 35 - 2 = 33 balls. The answer is 33.
|
||||
|
||||
Q: Olivia has $23. She bought five bagels for $3 each. How much money does she have left?
|
||||
A: She bought 5 bagels for $3 each. This means she spent 5
|
||||
|
||||
Q: When I was 6 my sister was half my age. Now I’m 70 how old is my sister?
|
||||
A:
|
||||
```
|
||||
|
||||
Output 1:
|
||||
```
|
||||
When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70 - 3 = 67. The answer is 67.
|
||||
```
|
||||
|
||||
Output 2:
|
||||
```
|
||||
When the narrator was 6, his sister was half his age, which is 3. Now that the narrator is 70, his sister would be 70 - 3 = 67 years old. The answer is 67.
|
||||
```
|
||||
|
||||
Output 3:
|
||||
```
|
||||
When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70/2 = 35. The answer is 35.
|
||||
```
|
||||
|
||||
Computing for the final answer involves a few steps (check out the paper for the details) but for the sake of simplicity, we can clearly see that there is already a majority answer emerging so that would essentially become the final answer.
|
Loading…
Reference in New Issue