The true cost of every call · Shipping AI Systems

A model call feels free the first time. You type a prompt, get an answer, and no one hands you a bill. The bill exists — you just haven’t seen it yet. As soon as your service handles real traffic, that invisible cost per call becomes the difference between a sustainable product and a monthly surprise. This section is how the cost is actually computed, how to estimate it before you spend it, and how to keep it honest as your traffic grows.

The pricing model

LLM providers price on tokens, not queries. A token is roughly 4 characters of English or 2-3 characters of Devanagari Nepali — but you should never guess. The provider counts precisely.

Every call has two components:

Input tokens — everything you send: system prompt, conversation history, retrieved documents, the user’s question.
Output tokens — what the model returns.

Different rates for each, and different rates by model. Rates as of late 2026:

Model	Input (per 1M tokens)	Output (per 1M tokens)
Claude Opus 4.7	$15.00	$75.00
Claude Sonnet 4.6	$3.00	$15.00
Claude Haiku 4.5	$0.25	$1.25
GPT-4.1	$2.50	$10.00
GPT-4.1 mini	$0.15	$0.60
OpenAI text-embedding-3-small	$0.02	—

Output is 4-5× more expensive than input. Long answers cost. Verbose system prompts cost. Repeated conversation history costs.

A concrete calculation

Take a Nepali policy Q&A bot using RAG:

System prompt: 400 tokens.
Retrieved documents (3 chunks × ~500 tokens): 1,500 tokens.
Conversation history (last 2 exchanges): 400 tokens.
User’s question: 50 tokens.
Total input: 2,350 tokens.
Answer: ~200 tokens output.

Using Claude Sonnet 4.6:

Input:  2,350 × $3  / 1,000,000  = $0.00705
Output: 200   × $15 / 1,000,000  = $0.00300
Per call:                          $0.01005  (≈ Rs 1.30)

Every call ≈ Rs 1.30. Not visible in isolation. Very visible at scale:

100 users × 10 calls/day × 30 days = 30,000 calls/month = Rs 39,000/month.
1,000 users × 10 calls/day × 30 days = 300,000 calls/month = Rs 390,000/month.

That is not a hypothetical. That is the actual month-end invoice at that traffic level, unless you controlled cost.

Where the cost hides

Three common places projects unintentionally spend more than needed:

1. Overly long system prompts.

A 2,000-token system prompt gets charged on every request. If you cut it to 400 tokens without losing quality, you save 80% of the system-prompt cost — which is often 30-40% of total cost. Pruning system prompts is one of the highest-ROI activities in an LLM product.

2. Growing conversation history.

Every new turn appends to the messages array. Turn 10 of a chat may have 5,000 tokens of history — and that history is sent to the model every turn. Techniques from Course 04 Chapter 3 (sliding window, summarisation) directly control this.

3. Unnecessarily verbose outputs.

If your product doesn’t need essays, ask for 3-sentence answers. max_tokens=200 on the API call is a hard ceiling — the model cannot exceed it. Set it. A default max_tokens=4000 when you only ever need 200 is a slow leak.

Model choice as a cost lever

The single biggest cost lever is which model you call. For a Q&A bot:

Model	Cost per call (2,350 in + 200 out)
Claude Opus 4.7	$0.050
Claude Sonnet 4.6	$0.010
Claude Haiku 4.5	$0.001
GPT-4.1 mini	$0.0005

Ten-fold cost differences. And for many tasks, Haiku 4.5 is nearly indistinguishable from Sonnet 4.6 — you often cannot tell the answers apart. The default assumption “I want the smartest model” is expensive and rarely necessary.

The right question is: what is the smallest, cheapest model that produces acceptable answers for my task? Benchmark this on your actual queries, not on a general leaderboard. Course 05 Chapter 4 covers exactly this evaluation discipline.

Estimating before you spend

Every serious project should have a spreadsheet — literally, before writing code — projecting monthly cost at expected traffic levels. Rows: input_tokens, output_tokens, model, price/M_input, price/M_output, calls_per_month, cost. Columns: pessimistic, expected, optimistic traffic.

Sample:

Assumptions:
- 1,000 daily active users
- 8 queries per user per day
- System prompt: 400 tokens
- Retrieved context: 1,500 tokens
- History: 400 tokens
- Query: 50 tokens
- Output: 200 tokens
- Model: Sonnet 4.6

Per call:
- Input: 2,350 tokens × $3/M = $0.00705
- Output: 200 tokens × $15/M = $0.00300
- Total: $0.01005

Per month:
- Calls: 1,000 × 8 × 30 = 240,000
- Cost: 240,000 × $0.01005 = $2,412 (≈ Rs 312,000)

If we switch to Haiku 4.5:
- Cost: 240,000 × $0.00095 = $228 (≈ Rs 30,000)
- Savings: 90%

That spreadsheet, one hour to build, is the difference between “I have a rough sense” and “I know exactly what next month’s invoice will be.”

Nepali-specific cost note

Devanagari text has a rougher tokenization rate than English — roughly 2-3 characters per token vs. 4 characters per token for English. A 500-word Nepali paragraph is often 700-800 tokens; the same paragraph in English is 500-600 tokens.

Practically: Nepali workloads cost 20-30% more per character than the English equivalent. Not a huge factor, but real. Include it in your spreadsheet.

Cost per user, not just cost per call

The metric that actually matters for product decisions is cost per user per month (CPM). If your monthly cost is Rs 300,000 and you have 3,000 users, that is Rs 100 per user per month. Compare this to what each user is worth to you (revenue, indirect value, strategic value).

A free tier that costs Rs 100/user/month and serves 100,000 free users is Rs 10 million/month. That is not a free tier — that is a runway drain. Understanding CPM up front lets you design the product (which models, which caching, which limits) to fit your economics, not the other way around.

Check your understanding

Quick check

—

A team writes a 3,000-token system prompt because 'more context makes the model behave better.' What are they paying, and what should they do?

Nothing — a longer prompt gives better answers.
They are paying for 3,000 input tokens on every single call. At 240,000 calls/month on Sonnet 4.6, that's $2,160 just for the system prompt (versus $288 for a pruned 400-token prompt) — a $1,800 leak per month. The fix is to write the shortest system prompt that produces the same quality, and to measure the effect on a fixed evaluation set. Long prompts have real, per-call, monthly costs; the discipline is to know that and act on it.
Try a smaller model.
Nothing — system prompts are free.

Quick check

—

You are choosing between Claude Opus 4.7 and Claude Haiku 4.5 for a policy Q&A bot. Opus 4.7 costs roughly 50× more per call. What is the correct decision process?

Always pick the most powerful model.
Build a fixed evaluation set of ~50 realistic policy queries with expected answers. Run both models on the same set, grade the answers on a rubric (correctness, completeness, faithfulness to sources). If Haiku scores within 5-10% of Opus on the tasks you care about, save the 50× cost and ship Haiku. If Opus is meaningfully better, model choice becomes a real trade-off between cost and quality — but the trade-off is now numeric, not vibes-based. Blindly defaulting to Opus is one of the most common budget mistakes in early LLM projects.
Always pick the cheapest.
Use both and average the answers.

What comes next

You know what each call costs. Next: how to make sure a user, a bug, or a viral moment can’t spend more than you can afford. Budgets, caps, and quotas — the hard limits that keep your service alive when something goes wrong.