System Prompts for AI Agents: Structure, Examples, and Best Practices

What a System Prompt Does

Every API call to a language model includes a system message, a user message, and optionally an assistant message representing conversation history. The system message is special because it sits at the top of the context window, receives the model's strongest attention, and persists unchanged across every turn of the conversation. While the user message changes with each interaction and the conversation history grows, the system prompt stays constant, forming the bedrock on which every response is built.

For standalone language model usage, the system prompt might be a single sentence: "You are a helpful assistant." For an AI agent, it is an entire operating manual. The system prompt tells the agent what role it plays, what tools it can use, what information it has access to, what it must never do, how to format its responses, what to do when it encounters something unexpected, and how to handle errors. The agent reads this document fresh on every call, which means it must be comprehensive enough to cover every situation without relying on the agent "remembering" anything from previous reads.

The reason system prompts matter so much for agents is that agents operate with autonomy. A chatbot can afford to occasionally misformat a response, because a human is reading it and can interpret the intent. An agent that calls tools, writes to databases, sends emails, or modifies code must be precise. A single misinterpreted instruction can cause the agent to call the wrong API, send a message to the wrong person, or execute an action the user did not authorize. The system prompt is where you prevent these failures, and the quality of that prevention determines whether the agent is trustworthy enough for production use.

The Anatomy of an Effective System Prompt

Production system prompts follow a consistent structure, not because there is a single correct format, but because the same categories of information recur across every agent type. Organizing these categories into distinct, labeled sections makes the prompt easier to write, easier to audit, and easier for the model to parse.

Identity and Role. The opening section establishes who the agent is and what it does. This is not decorative; it calibrates the model's behavior across everything that follows. "You are a customer support agent for Acme Software" produces meaningfully different behavior than "You are a general-purpose assistant." The identity section should include the agent's name if it has one, the organization it represents, and a one-sentence summary of its purpose. More detailed agents benefit from a brief paragraph explaining the agent's scope: what kinds of tasks it handles and, equally important, what kinds of tasks it does not handle.

Capabilities and Tools. This section lists every tool, API, database, and data source the agent can access, with clear descriptions of when to use each one. Each tool entry should include the tool's purpose, the situations that call for it, the inputs it expects, and the shape of the output it returns. Vague descriptions like "customer lookup tool" leave the model guessing; specific descriptions like "searches the customer database by email or customer ID, returns name, plan tier, billing status, and open support tickets as a JSON object" give the model everything it needs. This section is where most agent reliability problems originate, because a model that does not understand its tools will misuse them.

Behavioral Rules. This section defines how the agent should act, covering conversation style, decision-making priorities, and operational procedures. It includes rules like "always greet the customer by name if their name is available," "never promise a refund without checking the refund policy tool first," and "if the customer mentions legal action, immediately escalate to a human agent." These rules should be concrete and testable. Each rule should be specific enough that you could write a test case for it: given this input, the agent should do this specific thing.

Constraints and Safety. This section defines what the agent must not do, which is often more important than what it should do. It covers data the agent must never reveal (internal pricing tiers, customer data from other accounts, system architecture details), actions the agent must never take without approval (issuing refunds above a threshold, modifying account settings, deleting data), and topics the agent should decline or redirect (medical advice, legal counsel, financial recommendations in regulated contexts). Constraints should be stated positively where possible: "refer medical questions to a licensed professional" is more reliable than "do not answer medical questions," because the positive framing gives the model an action to take rather than just a prohibition to respect.

Output Format. This section specifies the shape of the agent's responses. For agents that pass output to downstream systems, this means JSON schemas, required fields, and enumerated values. For agents that interact with users, it covers tone, length, structure, and when to use lists versus prose. The format section should include examples of correctly formatted output, because models comply with format specifications far more reliably when they can see a concrete instance than when they are given abstract rules.

Error Handling and Fallbacks. This section tells the agent what to do when things go wrong: a tool returns an error, the user asks something outside the agent's scope, or the agent is not confident in its answer. Without explicit fallback instructions, agents tend to either hallucinate an answer or repeat the same failed action in a loop. Good fallback instructions say "if the customer lookup tool returns an error, apologize for the delay and ask the customer to verify their email address" rather than leaving the agent to improvise.

Writing Tool Descriptions That Actually Work

Tool descriptions are prompts within the system prompt, and they deserve dedicated attention because they directly control how the agent uses its capabilities. A model decides whether to call a tool, which tool to call, and what arguments to pass based entirely on the tool's name, description, and parameter schema. Getting these right eliminates the most common class of agent errors.

The tool name should be descriptive and unambiguous. Names like "search" or "get_data" tell the model nothing about what specifically gets searched or retrieved. Names like "search_customer_by_email" or "get_order_status" eliminate ambiguity. When an agent has many tools, clear names help the model distinguish between them without relying on the descriptions alone.

The description should answer three questions: what does this tool do, when should it be used, and what does it return? A good description reads like a concise manual entry. "Looks up a customer's account details by email address. Use this when the customer needs help with their account, billing, or subscription. Returns a JSON object with fields: customer_id, name, email, plan_name, plan_status, billing_cycle_end, and an array of recent_tickets." This gives the model clear criteria for when to use the tool and clear expectations for the output, both of which reduce errors.

Parameter descriptions matter as much as the tool description. If a parameter named "query" can accept either an email address or a customer ID, say so explicitly. If a parameter has valid values that the model might not know, list them. If a parameter is optional, explain when to include it and when to omit it. The less the model has to infer about your tools, the more reliably it will use them.

Negative examples are surprisingly effective for tools that the agent frequently misuses. If the agent keeps calling the refund tool when customers ask about their balance, add a note to the refund tool description: "Do not use this tool for balance inquiries. Use the account_balance tool instead." Negative instructions in tool descriptions have a higher compliance rate than the same instructions placed elsewhere in the system prompt, because the model reads them in the exact context where the decision is being made.

Common System Prompt Patterns in Production

Real production agents converge on a few proven patterns because they solve the same problems reliably.

The runbook pattern organizes the system prompt as a set of if-then rules, like a customer support runbook. "If the customer asks about pricing, call the pricing tool. If the customer asks to cancel, confirm the reason first, then call the cancellation tool. If the customer asks about a feature that does not exist, check the roadmap tool before responding." This pattern is highly testable and produces consistent behavior, because each rule is a specific instruction for a specific situation.

The persona pattern gives the agent a detailed character description that shapes its communication style. "You are Alex, a senior support specialist at Acme who has been with the company for five years. You speak in clear, professional language. You never use jargon without explaining it. You acknowledge customer frustration before offering solutions." Persona prompts work because they activate coherent behavioral clusters in the model rather than specifying each behavior individually. The model has seen thousands of examples of "senior support specialists" and can infer appropriate behavior from the archetype.

The tiered access pattern defines different behavior levels based on the context. "For basic plan users, answer questions about features and direct upgrade inquiries to the sales team. For enterprise users, provide detailed technical guidance and offer to schedule a call with the engineering team. For internal users, expose diagnostic information and allow direct database queries." This pattern is essential for agents that serve multiple audience types through the same interface.

The chain of thought gating pattern requires the agent to reason explicitly before taking action. "Before calling any tool, state what you intend to do and why. Before sending any customer-facing response, verify that it does not contain internal information." This pattern adds latency but dramatically reduces errors, because it forces the model to commit to a plan before executing, giving the orchestration layer an opportunity to intercept mistakes.

System Prompt Maintenance and Versioning

A system prompt is not a document you write once and forget. It evolves continuously as you discover new failure modes, add tools, change business rules, and respond to user feedback. Managing this evolution requires the same discipline you apply to any other critical system component.

Store system prompts in version control alongside your code. Each commit should have a clear message explaining what changed and why: "added escalation rule for billing disputes over $500" or "tightened output format to always include order_id in response." This history becomes invaluable when a behavior change introduces a regression and you need to identify which prompt edit caused it.

Test every change against a regression suite before deploying. The suite should cover the most common interaction types, the known edge cases, and the adversarial inputs that have caused problems in the past. A change that improves one behavior while degrading another is not a net positive, and you cannot catch that without running the full suite. Automated evaluation makes this practical by turning a thirty-minute manual review into a three-minute automated run.

Monitor prompt performance in production using metrics that reflect actual agent quality: task completion rate, tool call accuracy, escalation rate, and user satisfaction scores. These metrics tell you whether the current prompt version is performing well or degrading, and they provide early warning when a prompt change has unintended consequences that did not surface in testing. The feedback loop from production metrics back into prompt iteration is what keeps the agent improving over time rather than slowly drifting.