What is few-shot prompting?
Few-shot prompting puts a few input and output examples in the prompt so the model copies the pattern. Here is what providers advise and what research shows.
Few-shot prompting means putting a small number of worked examples, each an input and the output you want, in the prompt before the real input. The model infers the task from the pattern. Prompts with none are zero-shot. Google’s docs say “Prompts that contain a few examples are called few-shot prompts, while prompts that provide no examples are called zero-shot prompts.” Nothing is retrained. Researchers call it in-context learning.
The GPT-3 paper (Brown et al., May 2020) made the approach widely known. It showed that scaling a model to 175 billion parameters “greatly improves task-agnostic, few-shot performance”, with tasks and demonstrations given only as text, “without any gradient updates or fine-tuning”.
What the providers say
All three major providers recommend examples. They differ on how many, and on reasoning models.
| Anthropic | OpenAI | Google (Gemini) | |
|---|---|---|---|
| Stance | Examples are one of the most reliable ways to steer format, tone and structure | Show “a handful of input/output examples” | “We recommend to always include few-shot examples” |
| How many | 3 to 5 | A handful; its sample has three | No number; too many can overfit |
| Choosing | Relevant, diverse, edge cases | “a diverse range of possible inputs” | “specific and varied examples” |
| Format | Wrap in <example> tags, several in <examples> |
Usually in a developer message; the sample tags each example with an id |
Keep XML tags, white space and newlines consistent |
All read 2026-10-30. Two further notes from the docs:
- OpenAI’s reasoning-model guide says “Try zero shot first, then few shot if needed”, because reasoning models often do not need examples. If you add them, they should “align very closely with your prompt instructions”.
- Google says examples can replace instructions “if your examples are clear enough in showing the task at hand”.
Anthropic also notes that examples shape how Claude reasons when thinking is on. It suggests presenting each as a problem, the method to apply and the expected answer.
What research says matters in the examples
The GPT-3 paper is from 2020. Later work found that some intuitions about examples are wrong.
- Format matters more than correct labels. Min et al. (2022) found that “ground truth demonstrations are in fact not required”: randomly replacing labels cost little on classification and multiple-choice tasks, across 12 models including GPT-3. Demonstrations still help by showing the label space, the kind of input text and the format. This is a result on those task types, not a license for wrong examples.
- Order can swing results. Lu et al. (2021) report that order can separate “near state-of-the-art and random guess performance”. A good order for one model did not transfer to another. Their entropy-based ordering gave a 13% relative gain for GPT-family models across eleven classification tasks.
- Similar examples beat random ones. Liu et al. (2021) retrieved the examples most semantically similar to each test input, which “consistently outperforms the random baseline”. Picking examples per input from a larger pool is called dynamic or retrieval-based selection.
- More examples can keep helping. Agarwal et al. (2024) studied hundreds or thousands of examples as context windows grew, and saw “significant performance gains” over few-shot on many tasks. Inference cost grows linearly with the number of examples.
Choosing and testing examples
- Start from real inputs, including the hard and odd ones, since every provider asks for diversity.
- Keep the format of each example identical, and mark where the examples start and end.
- Start with 3 to 5, then change the count and the order and compare.
- Compare against a zero-shot prompt on the same cases. On reasoning models it may do as well.
Because order, count and example choice all move results, treat them as prompt changes to test, not settings to guess. Run each variant over a fixed set of test cases and grade the outputs, as described in LLM evaluation. Anthropic defines an eval as “a test for an AI system: give an AI an input, then apply grading logic to its output to measure success”.
Comparing example sets in HoneyHive
A HoneyHive experiment runs a function on a dataset with evaluate() and scores each output with evaluators. To compare example sets, run one experiment per variant (different examples, count or order) on the same dataset and compare the aggregated metrics.
FAQ
How many examples should a few-shot prompt have?
Anthropic's prompting guide says to include 3 to 5 examples. OpenAI's guide says "a handful", and Google's gives no number and warns that too many examples can make the model overfit to them. Test the count on your own task.
What is the difference between zero-shot and few-shot prompting?
A zero-shot prompt has instructions and no examples. A few-shot prompt adds a few input and output examples. Google's docs define both terms this way.
Is few-shot prompting the same as fine-tuning?
No. Few-shot prompting leaves the model's weights alone and shows the task in the prompt. The GPT-3 paper applied the model "without any gradient updates or fine-tuning".