The short version
If you want a model to produce something consistently, two or three good examples beat a paragraph of “please be sure to…”. But few-shot has a few counterintuitive edges, and hitting one leaves you worse off than not using it.
Counterintuitive 1: more examples isn’t better
Ten near-identical examples are worse than two or three that cover different situations. Repetitive examples burn context for nothing, and they anchor the model too hard on that single pattern.
The criterion for picking examples isn’t quantity, it’s stretching the boundaries: the most typical case, the one most likely to go wrong, and one where the right answer is empty or a refusal.
Counterintuitive 2: the model copies things you never meant to teach it
This is the sneakiest trap. The model doesn’t just learn the content of your examples. It takes every property they carry, including the ones you never noticed yourself:
- Your example answers all happen to be short → it learns “keep it short” and phones it in when a question deserves a real answer
- Your example inputs all happen to be one type → change the type and it generalizes badly
- One example contains an irrelevant verbal tic → now every response comes with it
When the output drifts, don’t rush to rewrite the instruction. Go back and look at what your examples have in common.
Counterintuitive 3: a counter-example beats a reminder
“Handle edge cases carefully” is filler. The model doesn’t know which cases are the edges.
Give it one concrete counter-example instead: “when the input looks like this, the correct behavior is to return empty.” That does more than ten reminders. Models are good at imitating and bad at inferring what you meant by “careful”.
Counterintuitive 4: teach format with an example, not a description
If you want a fixed format, skip the description of “separate with two newlines, bold the heading” and hand over one example already laid out that way. Show it, don’t tell it. (Same for structured output: a schema plus an example is always more reliable than prose.)
Wrapping up
Few-shot is fundamentally “defining what you want with samples”, and samples pass on the good and the bad faithfully. So the time is better spent picking examples than writing the prompt. While you pick, ask yourself one more question: taken together, what are these examples actually teaching the model?