Few-Shot Prompting: Teaching AI with Examples for Reliable Results

Why Examples Work Better Than Instructions

Language models learn by predicting the next token based on patterns in their training data. When you give the model an instruction like "extract the company name, revenue, and founding year from the following text and return them as JSON," the model must interpret your instruction, figure out what you mean by each field, decide on a JSON structure, and produce the output, all from a single abstract specification. When you instead show the model three examples of the exact transformation you want, it does not need to interpret anything. It sees the pattern and replicates it.

The difference is measurable. On the SuperGLUE benchmark for natural language understanding, GPT-3 went from 55 percent accuracy with zero-shot prompting to 71 percent with few-shot, a 16-point improvement from adding examples. On structured extraction tasks, the improvement is often larger, because format compliance is one of the areas where examples provide the most benefit. A model that receives format instructions without examples might return JSON 85 percent of the time. The same model with three well-crafted examples returns valid JSON 98 percent of the time or higher.

The mechanism is called in-context learning. Unlike fine-tuning, which changes the model's weights, in-context learning happens entirely within the forward pass: the model reads the examples, identifies the pattern, and applies it to the new input, all without any parameter update. This is why few-shot prompting is instant, costs nothing beyond the extra tokens, and works with any model through any API. It also means the learning is temporary, lasting only for the current context window, which is exactly what you want for flexible, task-specific behavior.

How Many Examples Do You Need

The research and practical experience converge on a clear answer: two to five examples cover the vast majority of use cases. One example is often enough to establish the format but leaves the model uncertain about edge cases. Two examples establish a pattern. Three to five examples cover enough variation that the model can generalize reliably. Beyond five, you hit diminishing returns, and the additional tokens increase cost without proportional accuracy gains.

The optimal number depends on the complexity of the task. Simple classification (positive/negative sentiment) works well with two examples. Structured extraction from messy text (pulling names, dates, and amounts from free-form paragraphs) typically needs four or five. Format transformation (converting one data structure to another) often works with three. The practical approach is to start with three examples, measure performance on your test suite, and add more only if accuracy on specific failure cases improves with additional examples.

There is a special case where more examples provide substantial gains: tasks with many possible output categories. If you are classifying support tickets into fifteen categories, three examples cannot cover all fifteen. In this case, you either provide more examples to cover the full range, or you combine few-shot with explicit instructions listing the valid categories. The combination of an exhaustive category list plus three representative examples tends to outperform either approach alone.

Selecting and Crafting Good Examples

The quality of your examples matters far more than the quantity. Poorly chosen examples teach the wrong patterns, and the model will faithfully replicate those wrong patterns on new inputs. Several principles guide example selection.

Cover the range. If the task has different input types, your examples should include at least one of each major type. For a customer support classifier, include one billing question, one technical issue, and one feature request, rather than three billing questions. The model generalizes from the variation it sees, so narrow examples produce narrow behavior.

Include edge cases. If there are inputs that are tricky, ambiguous, or commonly mishandled, include one as an example. A sentiment classifier that never sees a sarcastic example in its few-shot set will misclassify sarcasm consistently. An entity extractor that never sees an input with no entities will hallucinate entities when presented with empty inputs. Edge case examples are disproportionately valuable because they teach the model how to handle the cases that would otherwise fail.

Be consistent. Every example must follow the same format and apply the same rules. If one example capitalizes the output and another does not, the model receives a contradictory signal and will apply capitalization inconsistently. If one example includes an explanation and another does not, the model will sometimes explain and sometimes not. Consistency across examples is what turns them from individual demonstrations into a reliable template.

Use realistic data. Examples constructed from made-up data that does not resemble real inputs teach patterns that do not transfer. If your production inputs are messy paragraphs with typos and abbreviations, your examples should also be messy paragraphs with typos and abbreviations, not clean, perfectly formatted text. The closer your examples are to the actual input distribution, the better the model performs on real data.

Order matters. Models are sensitive to the order of examples. Research has shown that the order of few-shot examples can cause accuracy to vary by 10 to 15 percentage points on the same task. A good heuristic is to place the most representative, clearest example last, immediately before the actual input, because the model attends most strongly to the most recent context. Place more unusual or edge-case examples earlier in the sequence.

Few-Shot Prompting for Structured Output

One of the highest-value applications of few-shot prompting is getting consistent structured output from language models. When an agent needs to produce JSON, XML, CSV, or any other machine-readable format, examples are far more effective than format specifications alone.

The reason is that format specifications describe structure abstractly, while examples show structure concretely. The instruction "return a JSON object with keys name, amount, and date" leaves room for interpretation: should amount be a string or a number? Should date be ISO 8601 or a human-readable format? Should the object be wrapped in an array? An example eliminates all ambiguity: {"name": "Acme Corp", "amount": 15000.00, "date": "2026-03-15"}. The model sees the exact types, formatting, and structure and replicates them.

For complex schemas with nested objects, arrays, or conditional fields, examples become essential. Describing a schema with three levels of nesting in natural language is error-prone and hard for the model to follow. Showing two complete examples of the expected output structure makes the pattern immediately clear. The model essentially performs template matching on the examples and fills in the blanks with the new data.

When combining few-shot examples with the structured output features offered by APIs (like JSON mode in OpenAI or tool use in Claude), the examples serve as additional guidance within an already constrained output space. The API-level constraint ensures the output is valid JSON; the examples ensure it follows the specific schema and conventions you need. This combination produces the highest format compliance rates available, typically above 99 percent.

Few-Shot in Agent Systems

AI agents benefit from few-shot prompting in several specific ways that go beyond simple question answering.

Tool call demonstrations. Including examples of correct tool calls in the system prompt is one of the most effective ways to reduce tool calling errors. An example showing the input situation, the reasoning, the tool call with its arguments, and the interpretation of the result teaches the model the entire tool-use workflow for that tool. Two or three such demonstrations per tool dramatically reduce the rate of incorrect arguments, unnecessary tool calls, and misinterpreted results.

Response style calibration. Showing the agent examples of ideal responses sets the tone, length, and structure far more reliably than descriptive instructions. If you want the agent to respond in two concise paragraphs that acknowledge the customer's concern before providing a solution, include two examples that do exactly that. The model will match the length, structure, and emotional tone of the examples with remarkable fidelity.

Decision boundary examples. Agents make classification decisions constantly: is this a billing question or a technical issue? Should I escalate or handle it myself? Is this input safe or potentially malicious? Few-shot examples that sit on the boundary between categories, the cases where the right decision is not obvious, are where examples provide the most value. They teach the model where you draw the line, which is precisely the information that abstract rules fail to convey.

Dynamic few-shot selection. Advanced agent systems select examples dynamically based on the current input rather than using a fixed set. When a new customer query arrives, the system embeds it, searches a library of solved examples for the most similar ones, and injects those as the few-shot examples for the current call. This approach provides the maximum relevance per example token, because the examples are always close to the task at hand. It requires a library of labeled examples and a retrieval system, but the accuracy improvement over static examples is substantial, typically 10 to 20 percentage points on diverse task distributions.

Common Mistakes with Few-Shot Prompting

Using too many examples. More is not always better. Beyond five or six examples, the additional tokens consume context window space that could be used for conversation history or retrieved knowledge, and the marginal accuracy gain is minimal. Worse, very long few-shot sections can push the actual task input into a low-attention zone in the middle of the context, degrading performance on the task itself.

Examples that disagree with instructions. If your instructions say "always respond in formal English" but one of your examples uses casual language, the model will sometimes follow the instruction and sometimes follow the example. When instructions and examples conflict, examples usually win, because pattern matching is a stronger signal than rule following for language models. Audit your examples against your instructions to ensure they are perfectly aligned.

Homogeneous examples. Three examples of the same input type create a strong prior that all inputs look like the examples. When the model encounters an input that differs from the examples, it may force-fit it into the pattern rather than handling it appropriately. Diverse examples produce more robust behavior across the full range of inputs the model will encounter.

Neglecting negative examples. Showing the model what not to do can be as valuable as showing what to do. If the agent has a common failure mode, like including PII in its responses or calling a tool unnecessarily, an example that shows the correct handling of that situation (not including PII, not calling the tool) directly addresses the failure. Negative examples are most effective when paired with a brief note explaining why the demonstrated behavior is correct.