Error Handling in AI Tool Calls

Categories of Tool Calling Errors

Tool calling errors fall into five distinct categories, each requiring different handling strategies. Model generation errors occur when the model produces invalid tool calls, calling nonexistent functions, omitting required parameters, or generating malformed JSON. Validation errors occur when the arguments are syntactically valid but violate schema constraints like type mismatches, out-of-range values, or invalid enum selections. Execution errors occur when the underlying function fails during operation. External service errors occur when third-party APIs, databases, or other dependencies are unavailable or return unexpected responses. Interpretation errors occur when the model misunderstands a tool result and produces an incorrect or misleading response to the user.

Each category has a different root cause and a different optimal response. Model generation errors are best addressed by returning descriptive error messages that help the model correct its output. Validation errors should provide specific guidance about what constraint was violated and what the correct format or range is. Execution errors should distinguish between retryable failures (timeouts, rate limits) and permanent failures (invalid credentials, resource not found). External service errors should trigger fallback mechanisms or inform the user about temporary limitations. Interpretation errors are the hardest to detect and usually require human review or downstream validation.

Returning Errors to the Model

The most important principle of tool calling error handling is to return errors as tool results rather than throwing exceptions. When an error occurs at any stage, the application should format a clear error message and return it to the model as the tool result. This keeps the conversation flowing and lets the model decide how to proceed. An exception that crashes the agent loop leaves the user without any response. An error message returned to the model lets it retry, try an alternative approach, or inform the user about the issue.

Effective error messages have three components: what went wrong, why it went wrong, and what the model can do about it. "Error: invalid date format" is minimally helpful. "Error: the date parameter '2026-13-45' is not a valid date. Expected format is YYYY-MM-DD with valid month (01-12) and day values. Please provide a correct date." gives the model specific, actionable guidance for correcting the error.

Consistent error formatting across all tools helps the model develop reliable error handling behavior. A standard error format might include a "status" field ("error"), an "error_code" field (for programmatic handling), a "message" field (human-readable description), and an optional "suggestion" field (what the model should try next). When all tools use the same error format, the model learns to recognize and respond to errors predictably.

Retry Logic

Retry logic should distinguish between transient errors (which may succeed on retry) and permanent errors (which will fail the same way every time). Network timeouts, rate limit responses (HTTP 429), temporary service unavailability (HTTP 503), and connection resets are transient errors that warrant retry. Authentication failures (HTTP 401), authorization failures (HTTP 403), resource not found (HTTP 404), and validation errors are permanent errors that should not be retried because they will produce the same result.

Exponential backoff is the standard retry strategy for transient errors. The first retry waits 1 second, the second waits 2 seconds, the third waits 4 seconds, and so on, up to a maximum backoff interval. Jitter (adding a small random delay) prevents multiple agents from retrying simultaneously and creating a thundering herd effect. A maximum retry count (typically 3 to 5) prevents infinite retry loops.

The application can handle retries transparently (retrying before returning a result to the model) or explicitly (returning the error to the model and letting it decide whether to retry). Transparent retry is appropriate for transient errors that are likely to succeed within a few attempts. Explicit retry is appropriate when the model might want to modify the arguments, try a different tool, or inform the user about the delay.

Circuit Breaker Pattern

Circuit breakers prevent cascading failures by stopping calls to a service that is consistently failing. The circuit has three states: closed (normal operation, calls pass through), open (service is known to be failing, calls are immediately rejected without attempting execution), and half-open (after a cooldown period, a single test call is allowed through to check if the service has recovered).

When a tool fails more than a configurable threshold number of times within a time window, the circuit opens. All subsequent calls to that tool immediately return an error message like "This service is temporarily unavailable. The system will automatically retry in 60 seconds." After the cooldown period, the circuit enters half-open state and allows one call through. If it succeeds, the circuit closes and normal operation resumes. If it fails, the circuit opens again.

Circuit breakers are especially important for agent systems where a failing tool can cause the model to retry repeatedly, consuming tokens and time without making progress. Without a circuit breaker, a model might make 20 calls to a failing API, each timing out after 30 seconds, wasting 10 minutes and significant API costs before giving up.

Graceful Degradation

Graceful degradation ensures the agent provides value even when some tools are unavailable. Rather than failing entirely when a tool is down, the agent should fall back to alternative approaches: using cached data instead of live data, using a different tool that provides similar functionality, generating a response from model training data with an explicit accuracy disclaimer, or informing the user about what information is available and what could not be retrieved.

Fallback hierarchies define the degradation path for each tool. A primary data source might fall back to a secondary data source, then to cached data, then to model knowledge with a caveat. Each fallback level should be explicitly configured, not left to the model to improvise, because model-improvised fallbacks can produce confidently stated but inaccurate information.

Partial results are often more valuable than no results. If a multi-tool task successfully completes 4 out of 5 tool calls, the agent should present the information it has and clearly indicate what is missing: "I was able to retrieve your order history and shipping status, but the payment system is temporarily unavailable so I cannot show the payment details. Would you like me to try again or continue with the information I have?"