Chapter 03 · Section IV · 10 min read
3.4 Do-and-verify: why one agent should not grade its own work
The separation between the part that does work and the part that checks it. The most reliable single safety improvement you can make to any loop.
Ask any single agent — human or model — whether their own work is done and correct, and the answer is almost always “yes.” Not because they’re lying; because they can’t see the gap between what they meant and what they produced. The fix is structural: use one agent to do the work and a different agent to verify it. The verifier catches what the doer can’t see.
The problem in one line
A single agent’s output is grounded in whatever the agent believed while producing it. Its self-review is grounded in the same beliefs. Errors that stem from bad assumptions survive self-review, because the review is done by the same assumptions.
Concrete: a cycle that summarises tickets might invent a ticket ID. Asked “is this summary accurate?” the same cycle sees the ID it just wrote, has no reason to doubt it, and says “yes, accurate.” The mistake ships.
The structural fix
Split the cycle into two roles:
- Doer. Given the task, produces the work. Uses tools. Writes the output.
- Verifier. Given the task and the doer’s output, grades it against explicit criteria. Cannot rewrite the output — only accept, reject, or send back for revision.
Two model calls, not one. Two prompts. Ideally two different perspectives — the doer is optimised for producing; the verifier is optimised for skepticism. Some teams even use two different models for the roles, betting that two models will make different mistakes and agreement between them is a stronger signal than one model agreeing with itself.
What the verifier should check
The verifier needs an explicit checklist. Without one, its “review” is a vibes-based rewrite and it will drift toward safer, blander output.
Good verifier prompts look like:
You are the verifier. The doer produced the following output for the following task.
Task: {task}
Output: {output}
Check every one of these criteria and return a JSON object with pass/fail per criterion:
1. Every claim about a specific ticket cites a ticket ID that exists in the provided list.
2. No claim is made about a customer whose name does not appear in the source.
3. The summary is under 200 words.
4. No PII (email addresses, phone numbers, addresses) appears in the output.
Return: {
overall: "approved" | "rejected",
criteria: { "1": "pass" | "fail", ... },
notes: "one-line reason if rejected"
}The verifier isn’t asked “is this good?” It’s asked “does this pass these four specific tests?” That’s a job it can do.
What happens on rejection
Two options:
- Send back to the doer with the verifier’s notes and try again. Cap the number of retries so a stuck loop doesn’t burn infinitely.
- Escalate to a human. The verifier rejected; the loop can’t fix it; a human should decide.
Which one depends on stakes. For a loop that drafts internal Slack summaries, retry-and-retry is fine. For a loop that files invoices, one rejection should mean “stop and ask a human.”
When do-and-verify goes wrong
Three failure modes worth knowing:
1. The verifier is too loose. It accepts everything. Symptom: bad output ships anyway. Fix: sharpen the checklist. If the verifier isn’t rejecting at least occasionally, the criteria are probably too vague.
2. The verifier is too strict. It rejects everything. Symptom: the loop retries forever. Fix: loosen criteria that don’t actually matter, and cap retries so a stuck loop escalates instead of spinning.
3. The verifier uses the same context as the doer. If the verifier sees the doer’s chain-of-thought, it will absorb the doer’s assumptions and stop being independent. Give the verifier only the task and the output, not the internal deliberation.
Why one agent doing both is tempting (and wrong)
It’s cheaper. One model call instead of two. Half the tokens. Faster cycles.
For low-stakes loops — a live loop you’re watching, a run-until-done where the stakes are your own time — the trade may be fine. For anything unattended, especially anything that ships work to real people, the cost of a bad cycle usually swamps the cost of the second call.
Rule of thumb: if the cost of a wrong output (in time, money, trust, or blast radius) is more than 10× the cost of a verifier call, run the verifier. That threshold is met more often than people expect.
What about human review?
Human review is the ultimate verifier, and for high-stakes work it’s the right answer. Do-and-verify with two models is a first line of automated checking; it catches the majority of mistakes cheaply and quickly. The human sees only what the verifier accepted, so their attention lands where it matters.
The pattern that works:
- Doer produces work.
- Verifier grades it against a checklist.
- Verifier rejects → back to doer (with retry cap).
- Verifier accepts → post to a human review queue for high-stakes actions, or ship for low-stakes actions.
The verifier reduces the volume the human has to look at. The human catches what the verifier misses. Together they’re much stronger than either alone.
Check your understanding
Quick check
—