Cost
Each task is a whole run: several model calls and tool calls, not one request. Running each task more than once multiplies that.
Home · Agents · 11 · Agent evaluation
Concept card · Agents
Agent evaluation
What this gets you. Score your agent on whole tasks: what it left behind, and the steps it took.
Next on this goal: Multi-agent.
Agent evaluation is running your agent on a fixed set of tasks and grading three things: the outcome it left in your system, the steps it took, and how it stopped.
What changed
Nov 1: Compare a language-model judge with human labels on eight scripted replies, and decide whether that judge may gate a release.
Earlier updates
Oct 29: Practice had three scripted trials. Decide whether the agent or the grade is wrong, and whether a change ships. Oct 25: First card. One made-up task shows a reply that says done while the outcome check fails.
Know whether your agent finished the job, not only whether its last reply sounded right.
From Evaluation basics: before you trust a language-model judge, what do you compare it with?
Show the answer
Your own marks on examples you already graded. Evaluation basics.
1 What it is
An agent's last reply can say "Done" when nothing was done. Anthropic's guide to agent evals separates the transcript, every tool call and result in a run, from the outcome, the state left behind. Its example: an agent says a flight is booked, but the outcome is whether the booking exists. So a task in an agent eval has three parts: the input, what done looks like in your system, and the stop you expect.
Gemini Enterprise Agent Platform grades the final response and the trajectory, the sequence of tool calls, as separate scores. Grade the outcome with code when you can, the same rule Evaluation basics gives for replies.
Example: made up to show the idea.
The Support assistant answers from sample docs for Northline Desk, a made-up product. As in Agent loop, it has three tools: search the docs, look up an account, and file a ticket, which waits for a support person's OK. You grade the reply, then the three things a reply can't show.
Task: "My export is missing attachments." Done looks like: a cited answer from export.md, and one ticket waiting for approval. Expected stop: waiting for a person.
| Grade | What you check | Run 1 | Run 2 |
|---|---|---|---|
| Reply | Cites export.md, says attachments aren't in the CSV | Pass | Pass |
| Outcome | One ticket exists, status "waiting for approval" | Pass | Fail: no ticket exists |
| Steps | Searched the docs before filing; filed at most once | Pass | Pass (it never filed) |
| Stop | Waiting for a person | Pass | Fail: stopped as "done" |
2 Why it matters
Quality
The same task can pass on one run and fail on the next. Anthropic's guide gives two readings over several runs: at least one success in several tries, and success every time. A support agent that a customer meets once needs the every-time reading.
Risk
A run can reach the right answer by a wrong step, such as filing before searching. A grade on the steps catches that. A pass on your tasks is not a promise about tasks you didn't write.
3 How to apply · try today
Write three tasks for your agent. For each, write what done looks like as something your code can check, such as a record that exists or a field that changed. Run each task three times. For each run, mark the outcome, whether any act tool ran more than once, and how it stopped. Fix the task that fails most often first.
Common mistake
Grading the final reply and calling it the agent's score. A reply can say a ticket was filed when none exists. Check the outcome in your system first.
What to grade, and how
| You want to know | Grade | Grade it with |
|---|---|---|
| Did it do the job? | Outcome | Code that checks your system's state |
| Did it get there the right way? | Steps | An expected tool list. Strict order when order matters, or "these tools, any order" |
| Did it stop where it should? | Stop | The stop reason in your trace |
| Does it work every time? | Outcome, steps and stop, run again | Several runs per task; read "every one passed" |
Self-check
Your agent replies "I've filed the ticket." What tells you it actually did?
Self-check
A task passes on 2 of 3 runs. Customers meet this agent once each. Which reading should you use to judge it?
Self-check
The answer is right, but the agent filed the ticket before it searched the docs. Which grade catches it?
Example: made up to show the idea.
Scripted, not a live model. No request is sent, and nothing leaves this page.
Is your judge right?
Before a language-model judge gates a release, compare its judge verdict with the human label on a fixed set of replies. Mark every disagreement. A green score next to a bad reply is not enough.
Northline Desk is the made-up support desk from Glance. A language-model judge scores eight scripted replies. Each row already has a human label, pass or fail, written before the judge ran. Some rows agree. Some disagree.
Check what you learned
Before a language-model judge gates a release, what do you do?
Take eight replies your own product already gave. Write the human label on each one first, then the judge verdict on the same rows. Use the checklist. No live model, and nothing is sent.
Show a checklist
- A fixed set of about eight replies, with the human label written first.
- The same set scored by the judge, and each disagreement listed with a one-line reason.
- A decision before the judge gates a release: fix the judge, fix the labels, or grow the set.
- At least one row checks the outcome or the steps, not only the final reply.
- Three real failure traces are eval cases, or queued on Production evaluation.
- Calibration runs again when the judge model or the rubric changes, so you know what changed.
- The method is the proof. You are not shopping for a grader product.
- The judge is still ok to gate only after disagreements are reviewed and it matches the human label on the risky rows.
Before a judge gates a release, compare each judge verdict with the human label, mark every disagreement, and don't trust a green score on its own.
Turn failed runs into cases on the saved set: Production evaluation →
Name the failure in the trace: Failure modes →
Understand it: grade the run, not only the reply
A reply that says done is one grade. The outcome, the steps and the stop are the other three.
Words you'll see in eval tools
Anthropic's guide uses these names. A task is one job, with an input and what done looks like. A trial is one attempt at that task. A grader is the check that scores a trial. A transcript, also called a trace or a trajectory, is every tool call and result. The outcome is the state left behind. A harness is the code that runs the trials and records them.
Anthropic's guide to success criteria calls code-based grading the fastest and most reliable, and says to test model grading for reliability before you scale it. So check the outcome, steps and stop with code, and keep a judge for the reply.
Matching steps
Three ways to match the tools a run called, in alphabetical order. Google's Agent Development Kit scores the tool calls against an expected list with tool_trajectory_avg_score. By default it needs a 100% match. See Why evaluate agents. Gemini Enterprise Agent Platform can require an exact match, trajectory_exact_match, scored 1 or 0. See Evaluate Gen AI agents. LangSmith has four modes: strict order, any order, only these tools, and at least these tools (strict, unordered, subset and superset). See trajectory evaluations.
Use a strict order when the sequence matters, such as search before you file. Use any order when two orders are both fine. A strict match fails a good run that only swapped two reads.
Microsoft Foundry lists tool-call and task-adherence evaluators that score a run pass or fail.
Reset between runs
Each trial starts from a clean state. A ticket left by run 1 can make run 2 look like it filed one, when it only found the old ticket.
Reading several runs
pass@k is at least one success in k tries. pass^k is success on every one of those tries. Anthropic's guide works this case: if each try succeeds 75% of the time, three tries all succeeding is about 42%. That figure is theirs, not a result from this site. The at-least-once reading fits when you can keep one success, as with code you test before merging. The every-time reading fits a support agent a customer meets once.
Where the tasks come from
Start from failures in real traces. Failure modes names them. Production evaluation is where you keep the saved set and grow it from runs that failed.
As a gate
Run the set before a prompt, model or tool change. That is the same idea as the checks gate in Working with a coding agent: the change waits until the checks pass.
What it doesn't prove
NIST AI 600-1 asks, in MEASURE 2.3, that performance be measured "for conditions similar to deployment setting(s)". Your tasks are a sample of those conditions, not all of them.
Check yourself
Name the three grades for one run, and which one you check with code first.
Show the answer
Outcome, steps and stop. Check the outcome with code first. A reply that says done is not that check.
Questions
- What is agent evaluation?
- Agent evaluation is running your agent on a fixed set of tasks and grading three things: the outcome it left in your system, the steps it took, and how it stopped.
- Is this the same as Evaluation basics?
- No. Evaluation basics grades one reply. Agent evaluation grades a whole run: the outcome, the steps and the stop. A reply grade is one part of it.
- Do I need a model as the judge?
- Not for most of it. Outcome, steps and stop can be checked with code. Use a judge for the reply only when a rule can't decide, and test it first.