OCR + LLM for invoices, receipts, and bank statements · AI for Accountants

Walk into almost any practising accountant’s office in Kathmandu on the 25th of the month and you will see the same scene: a Dell tower from 2017, two stacks of paper a foot high, and someone keying invoice numbers from a NIC Asia statement into Tally one row at a time. This is the work that AI has, finally and quietly, made unnecessary — or at least made into a different job. The job is no longer typing. The job is checking, in the right places, what the machine has already typed for you. This section is about how to actually do that shift on Nepali source documents, where the paper is messy, the script is mixed, and “auto-post” is still a word you should be afraid of.

Printed Devanagari is solved. Handwriting is not.

The first thing to internalise is that OCR in 2026 is two different problems wearing one name. Printed Devanagari — the kind of text that comes off a thermal printer at a Bhatbhateni till, off the laser printer at a wholesaler in Birgunj, off the bank statement PDF from Global IME — is a solved problem. Modern multimodal models (Claude, GPT-class models, Gemini) read printed Devanagari, Roman Nepali, and English on the same page at character-error rates well under one percent. You can paste a screenshot of a Nabil bank statement into Claude and ask for a table, and the table will be right.

Handwritten Nepali is a different sport. The little blue receipt from the kirana pasal with रू. १२,४५०/- in a shopkeeper’s looping hand, the manual challan from a transporter, the cashier’s pencil note on the corner of a tax invoice — these are still where models break down, sometimes silently. Confidence on a handwritten amount looks the same as confidence on a printed one; the model will not always tell you it is guessing. If a document is mostly handwritten and the amount is material, do not run it through OCR at all. Type it. The minute you save is not worth the half-day you may spend later finding why the trial balance is रू. ९,००,००० short.

So the practical split, before you choose a tool: printed pages go to the model; handwritten pages of any material amount go to a human. Mixed documents — a printed invoice with a handwritten discount scribbled at the bottom — go to the model with an explicit instruction to flag the handwritten region for review.

What to extract, per document type

Different documents have different minimum fields you should pull. Be explicit with the model about what you want; do not leave it to choose.

1. Supplier invoice (VAT or non-VAT). Pull: invoice number, invoice date (in BS and AD where present), supplier name, supplier PAN/VAT number, line items with quantity and rate, sub-total, VAT amount (or a clear exempt flag), total, and any TDS noted. The model should propose a chart-of-accounts category but mark it as a proposal, not a posting. For VAT invoices the rule is non-negotiable: the 0.13 component must be separated from the net, on a per-line basis where the invoice mixes VAT and exempt items. Most teams’ worst posting errors come from a model that quietly applied 13% to a row that was actually exempt.

2. Customer receipt / counterfoil. Pull: receipt number, date, customer name, amount, payment mode (cash / bank transfer / cheque / Khalti / eSewa / FonePay), and reference number if any. The reference number is the value here — that is the field that lets you tie this receipt to the bank credit in the next section’s reconciliation step.

3. Bank statement PDF (e.g., NIC Asia, Nabil, Global IME, Siddhartha). Pull: each transaction’s date, narration verbatim, debit, credit, and running balance. Do not let the model “clean up” the narration — the narration is forensic. MBL/NPS/TXN-***/RAJU TRADERS is what lets you trace; Payment to Raju Traders is what loses you the trail. Ask explicitly for the narration field to be preserved character-for-character.

4. IRD challan and VAT return printouts. Pull: PAN, period (month / quarter), tax type, amount, date, challan / submission reference. These are reconciliation anchors, not transactions to post; treat the extracted data as something you will compare against the ledger.

5. Khalti / eSewa / FonePay wallet exports. These usually come as CSV already, so OCR is not the issue — the issue is that the merchant-side and the customer-side narrations do not match the bank-side narrations, and the model is good at proposing the bridge. Feed it both sides and ask which lines correspond.

The spot-check rule that actually works

You will not check every row. If you check every row, you have saved no time. The rule that gives you most of the safety of full review at a fraction of the cost is the top-and-bottom rule.

After the model has produced its extracted table, sort it twice. First, by absolute amount, descending — and human-check the top ten percent by value. The bookkeeping principle here is older than OCR: errors on large amounts matter more than errors on small ones. Second, sort by the model’s confidence score, ascending — and human-check the bottom ten percent. These are the rows the model itself is unsure about. The overlap between the two sorts is small but it is where almost all material errors live.

For documents with high total value or unusual line patterns — a single invoice over रू. १०,००,०००, anything to a new supplier, anything in a foreign currency — review the whole document, not a sample. The rule is a floor, not a ceiling.

A prompt pattern you can paste tomorrow

The shape of the prompt matters more than the model you use. A prompt that works across Claude, ChatGPT, and Gemini looks roughly like this — adapt the field list to the document type:

You are reading a Nepali supplier invoice (image attached). Extract the following fields and return them as a Markdown table: invoice_number, invoice_date_BS, invoice_date_AD, supplier_name, supplier_PAN, line_description, quantity, rate, line_amount, VAT_applicable (yes/no/exempt), VAT_amount, line_total, proposed_account, confidence (high/medium/low). One row per line item. Preserve any Devanagari text exactly as written. After the table, list any field you could not read with confidence and any handwritten region you saw. Do not infer values you cannot see.

Notice three things that this prompt does deliberately. It names the script (Nepali), so the model does not silently transliterate. It separates VAT from gross, which forces the model to make the call explicit rather than burying it. And it forbids inference — the last sentence is the most important one in the whole prompt, because the failure mode you want to avoid is a model that confidently fills in a PAN it cannot actually see.

The “never auto-post” rule

This is the rule that distinguishes a working AI workflow from a quiet disaster. No matter how high the confidence, the model’s output is a draft posting, not a posting. A human, with the chart of accounts and the firm’s context in their head, presses the post button.

There are two reasons this rule survives even as models get better. The first is legal: under NFRS for SMEs and IRD audit expectations, the firm — not its software — is responsible for the entries in its books. “The AI said so” is not a defence to a tax officer who has found a misclassified expense. The second is practical: the failure modes of LLMs are precisely the ones humans are bad at catching after the fact. A model that picks the wrong account but with a plausible name, on a thousand rows, will not produce a flag in the ledger; it will produce a slow, quiet distortion of the management accounts that no one notices until someone tries to use them for a decision.

The discipline is therefore not “review at the end of the month.” It is review at the moment the batch lands, before any row goes into Tally or Xero or Sage or whatever you keep your books in. Treat the LLM output the same way you would treat work from a smart but new junior — useful, fast, and absolutely requiring a senior eye before it goes anywhere near the general ledger.

Check your understanding

Quick check

—

You have a stack of 80 printed supplier invoices in Devanagari and English to process for the month. Which workflow is safest and most time-efficient?

Run all 80 through OCR + LLM, auto-post anything with high confidence, and manually re-key the rest.
Run all 80 through OCR + LLM with confidence scores, then human-review the top 10% by value and the bottom 10% by confidence before posting any row.
Manually re-key all 80 invoices because OCR is not reliable enough for VAT-bearing documents.
Run all 80 through OCR + LLM and post everything, since modern models read printed Devanagari at well under 1% error.

Quick check

—

A supplier invoice is mostly typed in Devanagari, but the discount line and the final total have been written in by hand by the supplier. The total is material. What is the right split between automation and manual work?

Send the whole document through OCR + LLM and trust the output — modern models handle Devanagari fine.
Skip the OCR step entirely and re-key the whole invoice manually.
Send it through OCR + LLM with an explicit instruction to flag the handwritten region, then have a human verify the handwritten discount and total against the paper before posting.
Send it through OCR + LLM but only use the line items; ignore the discount and total and let the system recompute.

What comes next

Extracting data from paper is only the first half of the bookkeeping shift. The second half is what category each row belongs in — and there a generic model, without your firm’s chart of accounts in front of it, will quietly do something worse than re-keying. The next section is on categorising transactions against the chart of accounts your firm actually uses.