Structured output: getting JSON back

Why JSON, specifically

JSON is the default because:

• Every programming language has a parser for it.
• The structure (keys and values) matches how prompts naturally describe fields.
• It’s easy to validate against a schema.
• LLMs, having been trained on enormous amounts of code and web data, produce it fluently.

You could ask the model for XML, YAML, or a custom format. Don’t. Every alternative is worse-supported by both the models and by every parsing library on earth. Use JSON.

The naïve approach and why it fails

The instinct is to ask nicely:

prompt = """Extract the fields from this receipt:
"KHALTI PAYMENT — Rs 1,240 at Bhatbhateni on 2026-06-15"

Return JSON with vendor, amount, date."""

response = client.messages.create(
    model="claude-haiku-4-5-20251001",
    max_tokens=200,
    temperature=0.0,
    messages=[{"role": "user", "content": prompt}],
)
data = json.loads(response.content[0].text)

This works most of the time. But “most of the time” is not “in production”, because sometimes the model returns:

Here's the JSON:

```json
{
  "vendor": "Bhatbhateni",
  "amount": 1240,
  "date": "2026-06-15"
}

Let me know if you need any changes!

Or:

```text
{
  vendor: "Bhatbhateni",
  amount: 1240,
  date: "2026-06-15",  // note: converted to ISO format
}

Or:

{"vendor":"Bhatbhateni","amount":"Rs 1,240","date":"June 15, 2026"}

Any of these will break json.loads or downstream code, and the failure mode is “5% of your users hit a bug you never see in local testing.”

The five habits for JSON output

The techniques that make structured output reliable:

1. Show the exact schema you want

"Return JSON with exactly these fields:

{
  "vendor":     string (the merchant name),
  "amount_npr": number (in Nepali rupees, integer if possible),
  "date":       string (YYYY-MM-DD format)
}

The model now knows every field name, every type, and every convention. Ambiguity is drastically reduced.

2. Say “return only the JSON”

"Return only the JSON object. No markdown code fences, no prose
before or after, no explanations."

This one instruction removes 80% of the “extra text around the JSON” problem. It’s not always enough (some models still add fences), so we defend downstream too.

3. Give a positive example

"For example, given the input 'ESewa Rs. 500 at 2026-05-10',
you should return:

{"vendor": "eSewa", "amount_npr": 500, "date": "2026-05-10"}
"

Few-shot examples, again. One example is usually enough for structured output; two is safer.

4. Define the failure case

"If the receipt is invalid or fields cannot be determined, return:

{"error": "parse_failed", "reason": "brief explanation"}
"

The error case is itself valid JSON, so downstream code has a single parsing path. Anything else that comes back is a bug in the prompt or the model.

5. Strip and parse defensively

Even with all four habits above, defend against stray markdown fences:

import json
import re

def parse_llm_json(text: str) -> dict:
    # Strip common markdown code fences
    text = text.strip()
    text = re.sub(r"^```(?:json)?\s*", "", text)
    text = re.sub(r"\s*```$", "", text)
    return json.loads(text)

This turns “model added markdown fences” from an outage into a non-event.

Anthropic’s dedicated support

Anthropic (and OpenAI, and others) have specific features for structured output:

Prefill — you can start the assistant’s response for it. If you prefill with {, the model is essentially forced to continue with JSON:

response = client.messages.create(
    model="claude-haiku-4-5-20251001",
    max_tokens=400,
    temperature=0.0,
    messages=[
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": "{"},   # prefill
    ],
)
# The response starts inside the JSON; prepend the opening brace when parsing
json_text = "{" + response.content[0].text
data = json.loads(json_text)

This is a robust technique — the model literally cannot start with markdown fences or prose, because you’ve already started the response with {.

Tool use — Anthropic’s tool-use API (Chapter 6) lets you declare a tool with a JSON schema, and the model’s output is guaranteed to conform. This is the strongest guarantee, but it’s more setup — we cover it properly in Chapter 6.

For most of Course 04’s examples we use the “prompt + defensive parser” approach because it works with any provider. When you know you’re on Anthropic, prefer prefill or tool use for absolute reliability.

Validation with pydantic

Getting JSON is not the same as getting correct JSON. A response with "amount_npr": "one thousand" is valid JSON but useless.

For any structured output that will be used programmatically, validate it with a schema library like pydantic:

from pydantic import BaseModel, Field, ValidationError
from typing import Optional
from datetime import date


class ReceiptFields(BaseModel):
    vendor: str = Field(min_length=1, max_length=200)
    amount_npr: int = Field(ge=0, le=100_000_000)
    date: date


def extract_receipt(text: str) -> Optional[ReceiptFields]:
    prompt = build_extraction_prompt(text)
    response = client.messages.create(...)
    raw = parse_llm_json(response.content[0].text)

    try:
        return ReceiptFields(**raw)
    except ValidationError as e:
        logger.warning(f"Validation failed for receipt: {e}")
        return None

Now you have three outcomes:

• Valid JSON, valid schema → typed ReceiptFields object, safe to use.
• Valid JSON, invalid schema → None, logged, downstream code handles.
• Invalid JSON → json.loads raises, caught by the same handler.

Every one of these is a known failure mode with a known response. The code stays boring.

A complete extraction pipeline

Putting it all together for the receipt case:

import json
import re
from datetime import date
from typing import Optional
from pydantic import BaseModel, Field, ValidationError
import anthropic

client = anthropic.Anthropic()


class ReceiptFields(BaseModel):
    vendor: str = Field(min_length=1, max_length=200)
    amount_npr: int = Field(ge=0, le=100_000_000)
    date: date


SYSTEM_PROMPT = "You are a receipt-parsing assistant. Return valid JSON only."

USER_PROMPT_TEMPLATE = """Extract fields from this Nepali/English receipt.

Receipt text:
{receipt_text}

Return only JSON with exactly these fields:
{{
  "vendor":     string (merchant name),
  "amount_npr": integer (in NPR, no commas or currency symbols),
  "date":       string (YYYY-MM-DD format)
}}

Example — given 'ESewa Rs. 500 at 2026-05-10', return:
{{"vendor": "eSewa", "amount_npr": 500, "date": "2026-05-10"}}

If the input is not a receipt, return:
{{"error": "not_a_receipt", "reason": "short explanation"}}

Return only the JSON object. No markdown, no prose."""


def parse_llm_json(text: str) -> dict:
    text = text.strip()
    text = re.sub(r"^```(?:json)?\s*", "", text)
    text = re.sub(r"\s*```$", "", text)
    return json.loads(text)


def extract_receipt(receipt_text: str) -> Optional[ReceiptFields]:
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=300,
        temperature=0.0,
        system=SYSTEM_PROMPT,
        messages=[{
            "role": "user",
            "content": USER_PROMPT_TEMPLATE.format(receipt_text=receipt_text),
        }],
    )
    raw = parse_llm_json(response.content[0].text)
    if "error" in raw:
        return None
    try:
        return ReceiptFields(**raw)
    except ValidationError:
        return None


if __name__ == "__main__":
    receipt = "KHALTI PAYMENT — Rs 1,240 at Bhatbhateni on 2026-06-15"
    fields = extract_receipt(receipt)
    if fields:
        print(f"Vendor: {fields.vendor}")
        print(f"Amount: NPR {fields.amount_npr}")
        print(f"Date:   {fields.date}")
    else:
        print("Could not parse receipt.")

Eighty lines. Handles empty input, non-receipt input, malformed JSON, and invalid field values. The happy path is three lines of usage code. This is the shape of every real LLM-powered extraction pipeline you’ll build.

Check your understanding

Quick check

—

A teammate's JSON extraction pipeline fails about 3% of the time in production because the model occasionally wraps output in markdown code fences. What are the two-part fix?

1. ANothing — 3% failure is normal for LLMs.
2. BFirst: strengthen the prompt with an explicit 'return only the JSON object, no markdown fences, no prose'. Second: add a defensive parser that strips leading and trailing ```json fences before calling json.loads. The prompt fix gets you most of the way; the defensive parser catches the last few. Together, the failure rate drops from 3% to near zero.
3. CSwitch to XML output.
4. DRetry any parse-failed call three more times.

Check answer→

Quick check

—

Your prompt reliably produces valid JSON, but one field, `amount_npr`, sometimes contains strings like 'one thousand' or 'Rs 500' instead of an integer. What's the right defence?

1. AAccept whatever the model returns.
2. BAdd pydantic validation with an integer type constraint on `amount_npr` (e.g. `amount_npr: int = Field(ge=0)`). Invalid values will now raise a `ValidationError` which your code catches and handles — typically by returning None or logging for later review. Also tighten the prompt: 'amount_npr must be an integer, no currency symbols, no words'. Prompt + validation is the pattern that keeps garbage from flowing downstream.
3. CConvert strings to integers with eval().
4. DDelete the amount field from the schema.

Check answer→

What comes next

You can get structured output out of an LLM. The last section of Chapter 2 handles the rest of production reality: errors, rate limits, and bad responses. Network calls fail; APIs return 429s; models sometimes just refuse. Building for these cases from day one is what separates apps that run for a year without paging you from apps that page you three times a week.