Chapter 03 · Section II · 10 min read
3.2 Self-review and self-critique
Asking the model to check its own work. When a second pass catches real mistakes, and when it just adds confidence to a wrong answer.
A useful trick: after the model produces an answer, ask it to critique that answer against specific criteria and produce a corrected version. This is self-review. It works — sometimes — and the “sometimes” is what this section is about.
The basic pattern
Two-turn version (works in a chat interface):
[Turn 1]
Extract every deadline from this contract as ISO dates.
<contract>...</contract>
[Turn 2]
Review the extraction above against these criteria:
1. Every date is a real deadline (not an effective date or a signing date).
2. Every date is in YYYY-MM-DD format.
3. No deadline was missed.
If you find any problems, output a corrected list. Otherwise, confirm.Single-prompt version (one shot):
Extract every deadline from the contract into a JSON array.
Then, in a second block, critique your own extraction against these criteria:
1. Every date is a real deadline (not an effective date or a signing date).
2. Every date is in YYYY-MM-DD format.
3. No deadline was missed.
Finally, output a corrected JSON array if the critique found problems.
<contract>...</contract>Both work. The single-prompt version is cheaper (one call, not two) but harder to iterate on because you can’t inspect the intermediate answer.
When self-review catches real mistakes
- Format checks. “Is every value a valid ISO date?” — mechanical, easy to verify, worth doing.
- Completeness checks against an explicit list. “Did the extraction include a value for every field in the schema?” — the model can count.
- Rule violations against a checklist you provide. “Does the summary contain any personal data?” — the criteria are yours, not the model’s, so the review is grounded.
When self-review is theatre
- “Is this the best answer?” The model will find something to critique and change something. That change is not necessarily an improvement — often it drifts toward more generic, safer text.
- Free-form critique with no criteria. “Review your answer” is close to “rewrite your answer to be different.” The output changes; whether it’s better is a coin flip.
- Factual review of things the model has no way to verify. If the model made up a statistic in the first pass, asking it to “double-check facts” won’t fix it — it has no external source to check against. See 3.3 on grounding.
Two-model review
If reliability really matters, run the critique with a different model — a stronger one, or a smaller one from a different family. Two models tend to make different mistakes, so agreement between them is a stronger signal than one model agreeing with itself. This is heavier and more expensive, but it’s the standard move for evals and grading tasks.
Making self-review actually work
- Give the reviewer a checklist, not a vibes prompt. “Check for X, Y, Z” not “look for anything wrong.”
- Ask for the corrected output, not just the critique. Otherwise you have to run a third prompt to apply the fixes.
- Log both passes. When you debug a bad answer later, you want to see whether the review missed the bug or the review caught it and the code ignored the correction.
- Sanity-check on an eval set (Chapter 5.1). If self-review doesn’t measurably raise accuracy on your task, don’t pay the tokens for it.
Check your understanding
Quick check
—