What an LLM API gives you (and what it doesn't) · Building with Language Models

Course 01 introduced the smallest possible LLM call. This course goes deep. Before we touch code again, an honest picture of what you actually get when you call Claude, GPT, or Gemini — what the API is really doing, what the model can and cannot do, and what constraints will shape every application you build. Skip this section and you will design against your assumptions; read it and you will design against reality.

What you get, concretely

A modern LLM API is a service that takes a piece of text (the prompt) and returns a piece of text (the response). Behind that simple contract sits a language model with hundreds of billions of parameters running on someone else’s GPUs. The model has been trained on trillions of tokens of text from the public internet, plus fine-tuned by the provider to be helpful, honest, and reasonably safe.

You send: a request over HTTPS with your prompt, some settings (temperature, max tokens), and your API key.

You receive: a response with the generated text, some metadata (how many tokens were consumed, whether the model stopped naturally or hit a limit), and — critically — a bill.

The three big providers as of 2026:

Anthropic — Claude family (Haiku, Sonnet, Opus). Known for careful reasoning and clean structured output.
OpenAI — GPT family (GPT-4o, GPT-4.1, GPT-5). Widest ecosystem, most third-party tools.
Google — Gemini family (Flash, Pro). Strong multimodal support, tight integration with Google Cloud.

For the rest of this course we use Anthropic’s Claude in examples, because the API is clean and the models are strong at multilingual work (including Nepali). Everything transfers to the other providers with minor renaming.

What the model can genuinely do

A short, honest list of things a modern LLM does well:

Translate competently between major languages, including Nepali ↔ English at high quality.
Summarise long text into shorter text, at whatever length you specify.
Rewrite text in a different style, tone, or register.
Extract structured information from unstructured text (a name from an address, an amount from a receipt).
Classify text against categories (spam/ham, sentiment, topic).
Answer questions when the answer is in general knowledge or in text you provide.
Reason through moderate multi-step problems (a math word problem, a policy decision).
Generate creative text — a story, a Nepali poem, marketing copy.
Write code in most mainstream programming languages.
Follow instructions carefully, especially when they’re specific.

If a task fits one of these families, an LLM is usually the right tool.

What the model cannot reliably do

The list of things LLMs are not good at is shorter and more important:

Compute exactly. They can approximate arithmetic but make surprising mistakes on complex calculations. Reach for a real calculator or code.
Access the internet or your files. By default, the model only sees what you put in the prompt. It cannot browse, cannot look at your database, cannot check today’s weather. Adding those capabilities (RAG, tool use) is what Chapters 5 and 6 are about.
Remember previous conversations. Every API call starts fresh unless you re-send the history. Chapter 3 covers memory.
Know recent facts. The training data has a cutoff — usually a few months to a year old. Ask about last week’s news and the model will confabulate or refuse.
Reliably follow rules 100% of the time. Even carefully prompted, models sometimes ignore instructions. Building robust systems requires assuming this happens and handling it.
Distinguish confidently between what they know and what they’re guessing. LLMs hallucinate — produce confident-sounding text that is factually wrong. This is the central failure mode you must design around.

Cost and latency, honestly

Two constraints that will shape every real application:

Cost. LLM providers price by tokens (roughly ¾ of a word each). Input tokens and output tokens have different prices. As of late 2025, Claude Haiku is roughly $0.25 per million input tokens and $1.25 per million output tokens; Sonnet is about $3 / $15; Opus is about $15 / $75. Prices trend downward but the relative pattern holds — smaller, cheaper models are excellent for most tasks, and you use the expensive models only when quality matters more than cost.

A typical short conversation call (a few hundred tokens in, a few hundred out) with Haiku costs a small fraction of a cent. A large RAG call (twenty thousand tokens of retrieved context in, five hundred out) with Sonnet costs roughly $0.07. A million small calls per month with Haiku costs on the order of a few hundred dollars. Budget accordingly.

Latency. Modern LLM APIs typically respond in 1-5 seconds for short prompts, longer for long ones. For interactive UIs, this is fast enough with a “typing…” indicator. For real-time systems (trading, phone calls) it is too slow — use smaller local models or narrower techniques.

The Nepali dimension

For Nepali applications specifically:

Understanding Nepali: Claude Sonnet and above are excellent. Claude Haiku is decent for simple tasks but occasionally misreads idiomatic Nepali. GPT-4o and Gemini Pro are comparable.
Producing Nepali: All top models produce fluent Devanagari Nepali. Occasionally they slip into English mid-response, which you can usually fix with a firmer prompt.
Romanised Nepali (“Nepglish”): Models handle it reasonably but sometimes get confused between phonetic variants. Explicit disambiguation in the prompt helps.
Nepal-specific facts: Coverage is uneven. Cricket, politics, and NEPSE are OK; district-level details are patchy. RAG is often needed for accurate local answers.
Cost: Nepali text uses roughly twice as many tokens as its English equivalent, because Devanagari characters tokenise less efficiently. Plan budgets accordingly.

This is real, not theoretical — it will affect every application you build for Nepal.

What this course covers

The rest of Course 04 is the practical craft of building with LLMs:

Chapter 1 — the basics: first call, temperature, cost.
Chapter 2 — writing prompts that survive contact with production; structured output; handling errors.
Chapter 3 — memory: making multi-turn conversations work; building a Nepali chatbot.
Chapter 4 — embeddings: turning text into vectors, searching by meaning.
Chapter 5 — RAG: grounding a model in your own documents.
Chapter 6 — a working assistant: design, tools, evaluation.

By the end you will have built a real Nepali AI assistant, evaluated it honestly, and made deliberate choices about its scope, safety, and limitations. Course 01 gave you a hello-world LLM call. Course 04 gives you a shippable product.

Check your understanding

Quick check

—

A teammate proposes building a Kathmandu-focused chatbot that answers 'what's the NEPSE close today?' by calling an LLM directly with no other tools. What's the fundamental problem with this design?

Nothing — modern LLMs know today's NEPSE close.
LLMs have training data cutoffs (often months old) and cannot access the internet or real-time data by default. Asked about today's NEPSE close, the model will either refuse or hallucinate a plausible-looking number. The fix is to fetch the actual value from a NEPSE feed and inject it into the prompt (Chapter 5's RAG-style pattern) or via function calling (Chapter 6), so the model reads real data rather than guessing.
Change to a bigger model.
Lower the temperature.

Quick check

—

A friend building a Nepali receipt-summariser wants to use Claude Opus for every call because 'it's the strongest model.' What's the practical response?

Opus is always the right choice for any task.
Opus is powerful but 60x more expensive per output token than Haiku. For a well-scoped task like receipt summarisation, Haiku is usually enough and lets the app be economically viable at scale. Start with the cheapest model that can do the job well; escalate to Sonnet or Opus only for cases where the small model demonstrably fails. This is standard 'model tier' design and can turn a $10,000/month app into a $200/month app with no user-visible quality difference.
Use a smaller local model instead.
Use two LLM calls per receipt for reliability.

What comes next

You know what an LLM API is and isn’t. The next section is the first real call — the same shape as Course 01 Chapter 5 Section 2, but with a careful walk through what each argument does, how tokens are counted, and what you’re being charged for. By the end of Section 2 you’ll be able to inspect any API call and predict what it will cost.