Classify Failure Modes: Reasoning, Tool, Context
A useful failure taxonomy turns a vague 'the agent broke' into an actionable fix. This topic teaches the three core agent failure modes — reasoning, tool, and context — and how to match observable signals to each so you fix the right layer.
Classify Failure Modes: Reasoning, Tool, Context
"The agent broke" is not actionable. To fix the right layer you need a small, shared vocabulary that maps each failure to a layer of the system. The minimum useful taxonomy is reasoning / tool / context, with safety as an orthogonal axis.
The three core modes
| Mode | Where it lives | Typical signal | Where the fix lives | | --- | --- | --- | --- | | Reasoning | Model's plan or inference | Right inputs, wrong conclusion or wrong plan | Instructions, planner prompt, model choice | | Tool | Tool selection, args, response handling | Wrong tool / wrong args / unhandled error | Tool schema, wrapper, instructions about tool use | | Context | What entered the prompt | Missing or wrong retrieval, stale memory, truncated history | Retrieval pipeline, memory layer, prompt assembly |
Safety failures (harmful content, leaked secrets, prompt-injection compliance) often co-occur with one of the three but are tracked on their own axis because their owner and threshold are different.
Diagnose by signal, not by guess
Pair each observable signal with the failure mode it indicates:
Match signals to failure modes
Drag each signal onto the failure mode it most directly indicates.
How the taxonomy changes the fix
- Reasoning fix: tighten the instructions, add a verification step, or change the model. Do not touch the retrieval pipeline.
- Tool fix: improve the tool's schema or description, add input validation in the wrapper, document failure handling in the instructions.
- Context fix: improve retrieval ranking, fix the memory write path, expand the prompt window for the missing material.
- Safety fix: tune content filters, add guardrails, harden against prompt injection — independently of whether the run was otherwise correct.
Exam framing: GH-600 questions love to describe a failure and offer four fixes from four different layers. The right answer is the one that matches the layer the signal points to, not the layer that feels most familiar.
Quick check
Quick check
An agent answers 'Your subscription renews on 5 May 2026' but the actual renewal date is 5 May 2027. Retrieval returned the correct customer record with the year 2027. Which failure mode is this?
Where this shows up on the exam
Two patterns dominate: (1) a short scenario plus four candidate fixes, asking which is correct — the answer is the fix at the layer the failure signal points to; and (2) a question that frames a failure as "hallucination" and asks for root cause — the answer is rarely "the model".
Key terms
- Reasoning failure
- The agent had the right tools and the right context but planned, ordered, or concluded incorrectly. Symptom: bad plan or wrong final inference despite good inputs.
- Tool failure
- The agent picked the wrong tool, passed wrong arguments, or mis-handled the tool's response (including ignoring an error). Symptom: tool span shows wrong args, wrong tool, or unhandled non-2xx.
- Context failure
- The agent did not have what it needed: missing retrieval, wrong document chunk, stale memory, dropped instructions, truncated history. Symptom: prompt lacks the fact the answer depends on.
- Safety failure
- Output violates content or policy constraints — harmful content, leaked secrets, prompt-injection compliance. Often surfaces alongside one of the three core modes.
- Failure-mode taxonomy
- A short, shared vocabulary the team uses to label every failure so trends are countable and fixes are routed to the right owner.
Common pitfalls
- Logging every failure as 'hallucination' — collapsing reasoning, tool, and context failures into one bucket destroys your ability to prioritise fixes.
- Fixing the model (new prompt, new model) when the trace says the failure was a missing retrieval or a wrong tool arg.
- Treating a tool 200 as success without checking the args; tool failures often masquerade as reasoning failures downstream.
- Ignoring safety failures because the run otherwise 'worked' — safety is a separate axis, not a degree of quality.