Chapter 03 · Section III · 17 min read
Why models fabricate citations — and the verification habit that catches it
The architectural reason language models invent confident citations, the prompt patterns that lower (never eliminate) the fabrication rate, and a thirty-second rule that turns verification from a chore into a reflex.
Up to this point the course has told you, repeatedly and from different angles, that AI models fabricate legal authorities. This section closes the loop. It explains, in plain language, why the fabrications happen — because once you see the mechanism, the verification habit stops feeling like a chore imposed on you and starts feeling like the obvious response to a known failure mode. It also gives you a prompt-engineering layer that reduces the fabrication rate, a thirty-second rule that turns verification into a reflex, and a frank statement about where the lawyer’s professional duty sits when the tool gets things wrong. The duty does not move. The tool is not a defendant.
Why models fabricate, in one paragraph
A large language model is, mechanically, a system that predicts what tokens are likely to come next given the tokens that came before. It is not a database of facts and it has no separate channel to truth. During training, it was rewarded for producing text that looked like the right kind of text in context — fluent, well-structured, plausibly authoritative. When you ask it for a Nepali Supreme Court decision on a point where its training data is thin or absent, the architecture does not pause and check a registry. It produces what the right kind of text would look like: a confident-sounding case name, a plausibly-shaped citation, a one-sentence holding that fits the question. The output is not a lie in any cognitive sense; the model has no cognition. It is a fluent imitation of the kind of answer the question seemed to call for, and the fluency is precisely what makes the fabrication dangerous.
The fabrications are, almost without exception, confident and well-formatted. That is the failure mode worth tattooing on the inside of your eyelids. They do not arrive with hedging language. They do not look like guesses. They look exactly like the real citations sitting next to them on the page, which is why the only defence is to verify every one of them against a source the model did not write.
The verification habit, codified
The habit is simple to state and harder to maintain at 10 p.m. before a 10 a.m. filing.
For every Act reference an AI model produces, open the source — the consolidated text on lawcommission.gov.np, or the official PDF you keep locally — and read the actual section before you quote, paraphrase, or rely on it.
For every case citation an AI model produces, look it up in Nepal Kanoon Patrika or on the Supreme Court of Nepal website. If you cannot find it, discard it. If you can find it, read the actual judgment to confirm that the holding is what the model said it was.
For every direct quotation an AI model attributes to a statute, regulation, or judgment, compare the quoted text against the source word by word. Models hallucinate punctuation, smooth over qualifiers, and silently substitute synonyms in ways that change meaning.
A prompt-engineering layer that helps — but does not solve
You can change how the model behaves with explicit instructions. These reduce the fabrication rate; they do not bring it to zero. Treat them as harm reduction, not as verification.
Tell the model, in plain language, not to invent. Open the prompt with: “If you do not know the answer or cannot identify a specific authority, say so clearly. Do not invent statutes, sections, cases, or citations. ‘I don’t know’ is an acceptable answer.” This single instruction, repeated, measurably lowers fabrication rates in current models. It does not eliminate them — the underlying architecture still rewards fluency — but it shifts the baseline.
Constrain the model to material you have provided. Instead of asking “what does Nepali law say about X,” paste the relevant Act yourself and ask “based only on the text I have provided, what does this Act say about X — and if the text does not address X, say so.” The model now has the actual document in its working context, which is a fundamentally different task from generating from memory. Fabrication rates fall sharply, though paraphrase errors remain.
Ask the model to flag low-confidence claims. “For each claim you make, indicate how confident you are and on what basis. If you are reciting from training data without a clear source, say so.” The model’s self-reported confidence is itself imperfect — models are sometimes confidently wrong about being confident — but the flag draws your attention to the right paragraphs to verify first.
Use a model with a retrieval layer when you can. Some legal-research tools combine a language model with a search index of actual authorities, so the model’s output is grounded in retrieved documents rather than generated from memory. This shifts the failure mode from invented citations to potentially mis-summarised real ones — better, but still requiring you to open the cited document.
The thirty-second rule
A working test that turns verification into a reflex.
For every legal claim a model produces — every Act reference, every section number, every case citation, every quoted phrase — ask yourself: can I verify this in thirty seconds? If the answer is yes (an Act on lawcommission.gov.np, a recent case on the Supreme Court website, a provision you know well enough to spot-check), always verify. The cost is trivial; the protection is total.
If the answer is no — the verification requires pulling a bound NKP volume, calling a court clerk, or paging through a long directive — then decide based on the stakes. For a research note that informs your own thinking, you may proceed with the model’s claim flagged as unverified and verify later. For anything that goes to a court, a client, a counterparty, or a regulator, you do not proceed without verification. Stakes set the rule.
The thirty-second rule has a useful secondary effect. It rewards working in an environment where verification is cheap — official statute portals open in a browser tab, NKP at your desk, the Supreme Court website bookmarked. The friction of verification is what determines whether the habit survives the urgent matter. Lower the friction in advance, while it is calm, and the habit holds when it counts.
The competence duty does not transfer
A closing observation, plainly stated, because it is the foundation of everything else in this chapter.
When you appear before a court, advise a client, or file a regulatory submission, the professional responsibility for what you say is yours. The Nepal Bar Council’s duty of competence is owed by you. The duty of candour to the court is owed by you. If a citation in your brief is fabricated, the fact that an AI model produced it is not a defence — it is, if anything, an aggravating circumstance, because the verification step that would have caught it is well known and freely available. “The model said so” is the legal equivalent of “the textbook said so” with the additional embarrassment that, unlike the textbook, the model produced something that does not exist.
This is not a reason to avoid AI in legal research. It is a reason to use it with the verification habit built into the workflow, not bolted on afterwards. The lawyers who get into trouble are not the ones who use AI; they are the ones who treat the model’s output as a finished product. The lawyers who get value from AI are the ones who treat the output as a draft to verify — and have the verification routine so deeply ingrained that an urgent matter does not displace it.
Check your understanding
Quick check
—Of the following prompt-engineering choices, which most reliably reduces the rate at which a general-purpose AI model fabricates Nepali statutes and case citations?
Quick check
—You are preparing a brief that will be filed in the Supreme Court tomorrow. A model has produced what looks like an excellent summary of a leading 2068 Supreme Court decision on your point. For purposes of verifying that the case exists and that the holding is as summarised, which source counts?
What comes next
This chapter has been about research — finding the right Act and the right case, and surviving the model’s confident fabrications along the way. The next chapter shifts to client work: how AI fits into the matters you handle for real people and real businesses, from intake through advice to delivery, and where the confidentiality, privilege, and competence questions actually bite when a chatbot sits between you and the file.