The practitioner's playbook · Shipping AI Systems

You have finished five courses. You know what AI is, how models learn, how data is built, how language models are wrapped into products, and how those products get shipped and stay healthy. This last section is a compact playbook — the specific checklists, thresholds, and habits from Course 05 in a form you can keep at your desk. Not a summary, a reference. The document that saves your future self time.

The day-one checklist

Before shipping any Nepal-facing AI product to real users:

Product & scope

Named user (specific person / role) and specific decision the product helps with.
Named scope in the system prompt (“I do X. I don’t do Y.”).
Refusal responses drafted for every out-of-scope category.

Infrastructure

Wrapped in a versioned API (FastAPI or similar).
Deployed to a managed platform with Singapore/Mumbai region.
Custom domain (.com.np preferred) with HTTPS.
Environment secrets stored via the platform (never in code).
Health-check endpoint (/health) returning 200.

Cost & limits

Provider-level monthly budget cap set (Anthropic Console / OpenAI Dashboard).
Per-user daily quota (calls + tokens).
Per-request cap on input length and max_tokens.
Rate limit per user or IP (10 req/min baseline).
Cost dashboard shows Rs per hour, per day, per month.

Reliability

Timeouts on every model call (30 s).
Retries with exponential backoff and jitter (for 429, 500-504).
Fallback path (cheaper model, or explicit “we’re down” message).
Graceful, honest user-facing error messages.

Logging & observability

Structured JSON logs on every request (request_id, user_id, latency, cost, model, prompt version).
No PII in plain logs (hash or mask).
Retention policy documented (30-90 days for ops; longer for audit).
Dashboard with latency p50/p95, error rate, cost rate, refusal rate.
Actionable alerts (5xx > 1%, cost > 3× baseline, latency p95 > 10 s).

Evaluation & feedback

Fixed eval set (50-200 items), covering Nepali + English + edge cases.
Daily automated eval run; results posted to team.
Thumbs up/down on every AI answer.
Weekly review of thumbs-down feedback (30-minute habit).
Backlog of failure categories, sorted by frequency × impact.

Versioning & rollouts

Pinned model version (claude-3-5-sonnet-20241022, not -latest).
System prompt versioned in code (v1.0, v1.1, …).
Index version tracked per RAG deploy.
Change log maintained (every deploy: what changed, on which axis).
Staging environment used before every production deploy.
Canary rollout for any non-trivial change (5% → 25% → 100%).
Documented rollback plan for each change.

Trust, privacy, and compliance

Product labelled as AI-generated in the UI.
Plain-language privacy policy (Nepali + English).
Consent flow for data collection.
Named human contact (“email me at nishan@example.np”).
Breach response policy documented.
Regulated-sector considerations reviewed (banking, health, legal).

Twenty-eight boxes. Each is a small decision. Together, they are the difference between a demo and a launched product.

The four-metric dashboard

The screen you keep open all day:

┌────────────────────────────────────────────┐
│ Latency:       p50 = ___ ms, p95 = ___ ms  │
│ Error rate:    ___% (past hour)            │
│ Cost:          Rs ___ / hour, Rs ___ / day │
│ Thumbs-down:   ___% (past 24h)             │
└────────────────────────────────────────────┘

Four numbers. Anyone on the team can read them in 30 seconds. When one shifts, investigate.

The weekly rhythm

Once you are shipping, an operational rhythm keeps the product improving:

Daily: 5 minutes on the dashboard. Any spikes?
Weekly: 30 minutes reading thumbs-down feedback. Categorise the failures.
Weekly: Automated eval run posted. Any drift?
Every deploy: Change log entry. Staged rollout. Metrics watched.
Monthly: 15 minutes on provider changelogs. Any drift coming?
Monthly: 30 minutes on the backlog. Which improvements to ship next?
Quarterly: Review the checklist. Anything decayed?

Two hours a week, roughly, for a well-run small AI product. Skipping it doesn’t save the time; it just moves the time from planned to reactive.

The five most common failure modes

Real failure patterns to protect against:

The viral wallet fire. Product goes viral; costs balloon. → Budget caps + quotas + rate limits (Chapter 2).
The silent quality drift. Model provider updates; behaviour changes. → Pinned models + continuous eval (Chapters 4-5).
The retrieval regression. Someone reindexes the RAG corpus with a new chunker; retrieval subtly breaks. → Index versioning + A/B tests (Chapter 5).
The user-trust break. Users discover the product hid something (data use, AI identity, silent failures). → Trust practices (Section 6.1).
The regulated-sector surprise. Product operates in a sensitive area; a specific rule is missed. → Sector-specific compliance (Section 6.2).

Every one of these has ended real projects. Every one has a specific defence in the checklist above. Do not deploy without them.

The one-page reference

The compressed version of Course 05, small enough to save:

╔════════════════════════════════════════════════════╗
║ SHIPPING AI SYSTEMS — 1-PAGE REFERENCE             ║
║                                                    ║
║ SCOPE                                              ║
║   Narrow. Specific user, specific decision.        ║
║   Refusals drafted for out-of-scope categories.    ║
║                                                    ║
║ INFRASTRUCTURE                                     ║
║   FastAPI + managed platform.                      ║
║   Singapore/Mumbai region. Custom .com.np.         ║
║   Health endpoint. Secrets in platform.            ║
║                                                    ║
║ COST                                               ║
║   Provider cap (monthly).                          ║
║   Per-user quota (daily calls + tokens).           ║
║   Per-request cap (input length + max_tokens).     ║
║   Rate limit (10/min user).                        ║
║   Cache repeated answers and static context.       ║
║                                                    ║
║ RELIABILITY                                        ║
║   Timeout 30 s. Retry 429/5xx with backoff.        ║
║   Fallback model. Honest user error messages.      ║
║                                                    ║
║ OBSERVABILITY                                      ║
║   Structured logs. No PII plain.                   ║
║   Dashboard: latency p95, errors, cost, thumbs.    ║
║   Alerts: 5xx>1%, cost>3x, thumbs-down spike.      ║
║                                                    ║
║ EVAL                                               ║
║   Fixed 50-200 item set. Daily auto-run.           ║
║   Nepali + English + edge + adversarial.           ║
║   LLM-as-judge with cross-provider grader.         ║
║                                                    ║
║ VERSIONING                                         ║
║   Pin model version. Version prompt in code.       ║
║   Version index. Change log per deploy.            ║
║   Staging first. Canary rollout. Fast rollback.    ║
║                                                    ║
║ TRUST                                              ║
║   Label as AI. Cite sources. Named human.          ║
║   Plain privacy policy, Nepali + English.          ║
║   Breach policy documented.                        ║
╚════════════════════════════════════════════════════╝

Print it. Pin it above your monitor.

Where you are now

You started Course 01 wondering what AI was. You leave Course 05 having:

Built with classical ML (Course 02-03).
Built with LLMs (Course 04).
Shipped a working service (Course 05).

That is a genuinely rare combination in Nepal in 2026. Most people in the country have used ChatGPT; very few can ship an AI system that runs reliably, serves real users, controls its costs, evaluates itself, and recovers from failure. You are one of the few.

What the practitioner does next

The end of a course is a beginning. Concretely:

Ship something small. A Nepali FAQ bot. A tool for your school. A helper for your NGO. Real users, real feedback, real learning. Nothing accelerates skill like a shipped product.
Join the community. Nepal Open Source Klub, Kathmandu Data Science Meetup, and the AI Literacy Nepal community are the places to share what you’re building and learn from others’ work.
Contribute back. Nepal-facing AI tooling is thin. Anything you build in the open — a Devanagari-friendly tokeniser, a Nepal-specific evaluation set, a bilingual UI component — is a gift to the next builder.
Keep learning. The frontier moves. Six months from now, some of what this course taught will be replaced by better patterns. Follow the practitioners you trust, read primary sources, run experiments.

The Nepal-specific closing thought

Nepal in 2026 is at an unusual inflection. AI tools are widely available; local products using them are scarce. Global funding is easy; Nepali distribution is hard. The productive question is not whether Nepali AI products should exist — they clearly should — but whose.

The answer, in part, is yours. This course was written on the premise that Nepali builders, informed by Nepali contexts, are the right people to build Nepal-serving AI. Not foreign labs’ Nepal-flavoured demos. Not one-size-fits-all Silicon Valley products.

You have the skills. The tools are affordable. The community is small enough to know. The audience is large enough to matter.

Ship something small. Then ship the next thing. Then the one after.

Course 05 close — and Track 03 close

That is the end of Course 05.

That is also the end of Track 03 — Building AI.

Five courses. Ninety sections. From “what is AI” to “shipping AI systems that serve real users in Nepal.” You are a practitioner now.

Track 03 is the technical build spine of AI Literacy Nepal. There are more tracks — Learning AI (understanding), and the Professionals tracks (apply AI to teaching, HR, accounting, marketing, law) — each with their own audience and their own value. Choose what matters next for you.

The work of AI in Nepal is early. Build well.