Types of AI Agent Learning: Feedback to Experience

How to Categorize Agent Learning

The clearest way to organize the types of agent learning is along two axes: where the change lives, and what drives it. The "where" axis runs from the context window, through external memory, to the model weights. The "what drives it" axis runs from raw examples, through explicit feedback, to outcome-based reward. Every learning method occupies a position on this map, and its position predicts its strengths and limitations.

The five types below progress roughly from cheapest and most temporary to most expensive and most permanent. This ordering is not a quality ranking; the cheaper methods are often the right choice. It is a guide to the natural sequence in which teams adopt them, starting with the methods that deliver the most improvement for the least risk and adding the heavier machinery only when the data and the task justify it.

In-Context Learning

In-context learning is the model's ability to adapt to instructions and examples placed directly in its prompt, with no change to its weights. When you show the model three examples of a desired output format and it produces a fourth in the same style, it has learned the pattern for the duration of that prompt. This is the fastest possible form of learning, taking effect the instant the context changes and costing nothing beyond the tokens consumed.

Its defining limitation is impermanence. Whatever the model learns in context vanishes when the session ends or when the relevant text scrolls out of the window. In-context learning is also bounded by the size of the context window: you can only fit so many examples and instructions before you run out of room. Despite these limits, it is the most heavily used learning mechanism in practice, because prompt engineering, few-shot examples, and retrieved context all operate through it. It is the layer where experimentation happens, because changes are instant and reversible.

Memory-Based Learning

Memory-based learning makes in-context learning persistent by storing information outside the model and retrieving it when relevant. The agent writes observations, user preferences, corrections, and outcomes to an external store, most often a vector database for semantic recall or a structured database for facts and state. On future tasks it retrieves the relevant entries and places them into context, recreating the in-context learning effect from accumulated experience.

This type is the backbone of practical agent improvement because it delivers persistence without the cost and risk of retraining. The agent's knowledge grows as its memory grows, while the model stays fixed. Retrieval-augmented generation is the most prominent example, letting an agent answer from a knowledge base that can be updated continuously. The quality of memory-based learning depends entirely on retrieval: information the agent never surfaces at the right moment might as well not exist, which is why retrieval design is as important as storage. The boundary between memory and genuine learning is subtle enough to deserve its own detailed treatment.

Supervised Fine-Tuning

Supervised fine-tuning is the first type that changes the model's weights. It works by collecting a dataset of input-output pairs that demonstrate correct behavior, then training the model to reproduce those outputs. For agents, the inputs are tasks and the outputs are successful completions, often drawn from the agent's own best runs or from human-authored examples.

Fine-tuning bakes patterns permanently into the weights, which makes the behavior automatic and removes the need to carry examples in context, reducing cost and latency at inference time. The trade-offs are significant: it requires a substantial volume of high-quality examples, typically hundreds to thousands, it takes a training run to produce and evaluate, and it carries the risk of degrading capabilities the model already had. Lightweight approaches such as adapter tuning, including LoRA, reduce the cost and risk by training a small set of additional parameters while leaving the base model untouched. Fine-tuning is the right choice when a task is stable and you have accumulated enough verified examples to make the patterns worth making permanent.

Reinforcement Learning from Feedback

Reinforcement learning from feedback trains the model not on examples of correct outputs but on signals about which outputs are better. The best-known form is reinforcement learning from human feedback, in which humans compare pairs of outputs, those comparisons train a reward model, and the reward model guides the optimization of the agent's policy. Direct preference optimization achieves a similar result more simply by training directly on preference pairs without a separate reward model. Reinforcement learning from AI feedback substitutes a capable model for the human rater to scale the process.

This type excels where correctness is a matter of judgment rather than a single right answer, such as tone, helpfulness, safety, and style. Because it learns from relative preferences rather than absolute labels, it can capture nuances that are hard to specify as explicit examples. It is also the most technically demanding type, requiring careful reward design to avoid the agent optimizing the measured signal at the expense of the real goal. The full mechanics of turning human judgments into model improvements are covered in learning from human feedback.

Experience-Based Self-Improvement

Experience-based learning turns the agent's own task trajectories into the source of improvement. Every attempt the agent makes produces a trace: the plan, the tool calls, and the outcome. Successful traces become positive training signal and failures become negative signal. The agent, or the system around it, uses these traces to fine-tune the model through supervised learning on successes or reinforcement learning from outcomes, so that future attempts benefit from past ones.

This is the most autonomous type, because the learning signal comes from the agent's own behavior rather than from human-provided examples or preferences. It is also the most dangerous if implemented carelessly, because learning from unverified self-generated data can compound errors into model collapse. The safeguard is to verify outcomes before they become training data, using grounded signals such as whether code passed its tests or whether a task achieved its goal. Done well, experience-based learning lets an agent improve at a task simply by doing it many times, which is the closest current systems come to the intuitive notion of learning from experience. The approach is explored fully in learning from experience.

Choosing the Right Type for Your Goal

The five types are not competitors; a mature agent uses several together. But for any specific improvement goal, one type is usually the right starting point, and matching the goal to the type avoids wasted effort.

If your goal is to change behavior quickly or to test an idea, use in-context learning, because it is instant and reversible. If your goal is to make the agent remember facts, preferences, or corrections across sessions, use memory-based learning, because it persists without retraining. If your goal is to make a stable, well-understood behavior faster and cheaper by removing it from the prompt, use supervised fine-tuning, once you have enough examples. If your goal is to align the agent with subtle quality judgments like tone and helpfulness, use reinforcement learning from feedback. If your goal is to have the agent get better at a repeatable task through sheer repetition, use experience-based self-improvement with verified outcomes.

The most common mistake is reaching for fine-tuning or reinforcement learning before exhausting the cheaper types. Memory and prompt improvements deliver the majority of achievable gains in most deployments, at a fraction of the cost and with none of the retraining risk. The heavier types earn their place only once the lighter ones have been pushed to their limits and the data exists to support them.

Combining Types in a Single Agent

In a mature agent the five types are not alternatives but layers that operate together, each handling the part of improvement it is best suited to. A single well-built agent might use in-context learning to follow the instructions in its current prompt, memory-based learning to recall what it learned about this user last week, supervised fine-tuning to handle its most common task type efficiently, reinforcement learning from feedback to maintain the tone its users prefer, and experience-based self-improvement to get steadily more reliable at a verifiable core task.

These layers reinforce one another. Memory and in-context learning surface the patterns that, once proven stable, become candidates for fine-tuning. Feedback collected at runtime supplies the preference data that reinforcement learning consumes. Verified experience accumulates into the dataset that supervised fine-tuning draws on. The runtime layers act as a discovery system for what works, and the training layers make the proven discoveries permanent, so the cheap fast mechanisms continuously feed the expensive slow ones.

Designing this combination is mostly about sequencing and division of labor. Decide which behaviors must change instantly and assign them to context and memory. Decide which behaviors are stable enough to bake in and assign them to training. Keep the boundary deliberate, because a behavior that is still being figured out belongs at the runtime layer, and only a behavior that has been validated belongs in the weights. An agent that gets this division right improves faster and more safely than one that leans on any single type alone.