Wrapping a model in an API

FastAPI in five lines

FastAPI is the modern standard for Python HTTP services. It is fast, it is small, and it has good defaults. Install:

pip install fastapi uvicorn anthropic python-dotenv

The smallest possible service:

# service.py
from fastapi import FastAPI

app = FastAPI()

@app.get("/hello")
def hello():
    return {"message": "Namaste from the API."}

Run it:

uvicorn service:app --reload

Open http://localhost:8000/hello in a browser. It responds. That is a service — a small program that listens for HTTP requests on a port and returns responses.

The --reload flag makes it restart on code changes; drop it in production.

Adding the model

Now the real thing: an endpoint that accepts a Nepali paragraph and returns a summary.

# service.py
import os
from fastapi import FastAPI
from pydantic import BaseModel
from anthropic import Anthropic
from dotenv import load_dotenv

load_dotenv()
app = FastAPI()
client = Anthropic()


class SummariseRequest(BaseModel):
    text: str
    max_sentences: int = 2


class SummariseResponse(BaseModel):
    summary: str
    input_length: int


@app.post("/summarise", response_model=SummariseResponse)
def summarise(req: SummariseRequest):
    prompt = (
        f"Summarise the following text in {req.max_sentences} sentences. "
        f"Keep the summary in the same language as the input.\n\n"
        f"Text:\n{req.text}"
    )

    response = client.messages.create(
        model="claude-3-5-sonnet-latest",
        max_tokens=300,
        messages=[{"role": "user", "content": prompt}],
    )

    return SummariseResponse(
        summary=response.content[0].text,
        input_length=len(req.text),
    )

Test from the command line:

curl -X POST http://localhost:8000/summarise \
  -H "Content-Type: application/json" \
  -d '{"text": "काठमाडौँ नेपालको राजधानी हो। यो शहर बागमती प्रदेशमा पर्दछ र नेपालको सबैभन्दा ठूलो शहर हो। यहाँ २५ लाख भन्दा बढी मानिस बस्दछन्।", "max_sentences": 1}'

Response:

{
  "summary": "काठमाडौँ बागमती प्रदेशमा अवस्थित नेपालको राजधानी र सबैभन्दा ठूलो शहर हो जहाँ २५ लाख भन्दा बढी मानिस बस्दछन्।",
  "input_length": 138
}

Twenty lines. A working AI service.

Why every piece matters

Let’s slow down and understand what each piece actually does.

BaseModel and Pydantic. FastAPI uses Pydantic to define the request and response shapes. Any request whose JSON does not match SummariseRequest is rejected with a 422 error automatically — before your code runs. You get validation for free.

Typed inputs. text: str means the API refuses non-string inputs. max_sentences: int = 2 means the field is optional (defaults to 2) but must be an integer if given. This surface-area constraint prevents whole classes of bugs.

response_model. Declaring the response type has two effects: it validates what you return, and it generates automatic OpenAPI docs. Visit http://localhost:8000/docs and you’ll see an interactive Swagger UI where anyone can test the endpoint. This is FastAPI’s superpower.

One responsibility per endpoint. /summarise does one thing. When you add features (translate, extract entities, classify), each becomes its own endpoint. This keeps the code readable and each URL testable in isolation.

Handling errors properly

The naive version above fails ungracefully when the model call fails. Fix:

from fastapi import HTTPException
from anthropic import APIError

@app.post("/summarise", response_model=SummariseResponse)
def summarise(req: SummariseRequest):
    if not req.text.strip():
        raise HTTPException(status_code=400, detail="Text cannot be empty.")

    if len(req.text) > 10_000:
        raise HTTPException(status_code=400, detail="Text is too long (max 10,000 chars).")

    try:
        response = client.messages.create(
            model="claude-3-5-sonnet-latest",
            max_tokens=300,
            messages=[{"role": "user", "content": _build_prompt(req)}],
        )
    except APIError as e:
        raise HTTPException(status_code=502, detail=f"Model provider error: {e}")

    return SummariseResponse(
        summary=response.content[0].text,
        input_length=len(req.text),
    )

Three things now:

1. Validate boundaries. Empty and oversize input are rejected with clear 400 errors before any model call happens.
2. Convert provider errors. An Anthropic API failure becomes a clean 502 for the caller — they don’t see stack traces.
3. Fail loud, fail clean. Errors are HTTP status codes with descriptive detail. No half-answers, no confused clients.

This one small pattern — validate → try → catch → convert — is the shape of every well-behaved endpoint.

Rate limiting, briefly

A public endpoint that calls an LLM without rate limits is a wallet on fire. The simplest defense:

from slowapi import Limiter
from slowapi.util import get_remote_address
from slowapi.errors import RateLimitExceeded

limiter = Limiter(key_func=get_remote_address)
app.state.limiter = limiter

@app.post("/summarise", response_model=SummariseResponse)
@limiter.limit("10/minute")
def summarise(request: Request, req: SummariseRequest):
    ...

Ten requests per minute per IP. A real production service would want per-user (once you have auth), per-plan, and per-endpoint limits — but at day 1, this stops the wallet from burning.

We’ll come back to costs and rate limiting properly in Chapter 2.

The service, running

Your directory now looks like this:

my-service/
├── service.py        # the code above
├── .env              # OPENAI_API_KEY, ANTHROPIC_API_KEY
├── requirements.txt  # fastapi uvicorn anthropic ...

Run in development:

uvicorn service:app --reload --port 8000

For production, drop --reload and let a supervisor (systemd, Docker, or a platform like Fly.io) keep the process alive:

uvicorn service:app --host 0.0.0.0 --port 8000 --workers 4

--workers 4 spawns four worker processes to handle concurrent requests — Python’s GIL means a single worker serves one request at a time, but four workers let you serve four in parallel. On modest hardware, this is enough for tens of requests per second, which is enough for a small Nepal-scale service.

What we’ve built

Twenty lines of production-shaped Python. It:

• Runs continuously as a process (not a notebook).
• Accepts HTTP requests from anywhere.
• Validates input at the boundary.
• Calls a model and returns a response.
• Handles errors cleanly.
• Rate-limits abuse.
• Has automatic API docs.
• Can be scaled to multiple workers.

That is a real service. Not a big one, not a fancy one, but real. The next section is where it goes — how to get this code running on a computer other people can reach.

Check your understanding

Quick check

—

Why is Pydantic (via BaseModel) the standard for input handling in FastAPI services?

1. AIt looks nice.
2. BIt provides declarative, type-checked validation of every request before your code runs. Bad input (wrong types, missing fields, out-of-range values) is rejected automatically with a clear 422 error, and your endpoint code only sees well-formed data. This eliminates entire classes of bugs and defensive checks, and the same schemas generate the automatic OpenAPI docs at /docs. Getting typed input right at the edge is one of the highest-leverage decisions in an API server.
3. CIt's required by FastAPI.
4. DIt generates database migrations.

Check answer→

Quick check

—

Your service handles Nepali paragraph summarisation. A user starts sending 5MB paragraphs. What is the correct response?

1. AJust process them; users know what they need.
2. BReject at the boundary with a clear 400 error and a friendly message. Oversized input costs money to process, ties up worker time, and often reflects abuse or misuse. Setting a documented max size (e.g. 10,000 chars) protects the wallet, the workers, and the user's own experience (they get a clear error instead of a five-second wait for a bad answer). The exact limit depends on the use case, but every LLM-facing endpoint should have one.
3. CSilently truncate the input to 1MB.
4. DCharge them extra.

Check answer→

What comes next

The service works on your laptop. That is the tiny world. Section 1.3 is the middle-sized world: getting it running on a computer that everyone can reach — a real deployment, on real infrastructure, addressable by a real URL. Every AI product’s day-one hurdle, made honest.