Why a model is only as good as its data · Working with Data

The single most repeated, most ignored, most expensive lesson in modern AI is this: the model is downstream of the data. You can have the largest, most modern, most beautifully tuned model in the world — and if the data behind it is wrong, biased, or unrepresentative, the model produces nonsense, confidently, at scale. This entire course exists because almost everyone who learns to build with AI tries to skip past data work, and almost everyone who does pays for it later.

Garbage in, confident garbage out

A model is a machine that maps inputs to outputs. It learns that mapping from examples. If you show it ten thousand examples of “loan applications and whether they defaulted”, it will learn something. What it learns depends entirely on what you showed it.

If your loan dataset comes only from Kathmandu, the model has no idea what defaults look like in Doti. If your training labels confused lakhs and crores in 0.4% of rows, the model has no idea which dimension it is working in. If your data was collected only from 2024 — a year of stable monsoons — the model has never seen a drought-year default, and will be wrong the first time it meets one.

None of these are bugs in the model. They are the model faithfully reflecting the data it was given.

Three real ways data sinks a project

A short, honest list of the failure modes you will recognise from your own work — or from headlines — within six months of building.

Representation mismatch. The training data does not represent the population the system will be used on. The facial recognition trained on European faces, deployed in Kathmandu. The medical AI trained on US patients, used on Nepali ones. The Nepali NER (named entity recognition) trained on Kathmandu Valley names, deployed on Karnali Province names. The model is doing its job; the dataset failed it.

Label noise. The “ground truth” labels are wrong. A loan dataset where defaults were marked by a tired clerk who confused “approved with notes” and “approved with conditions”. A medical dataset where some patients labelled “no diabetes” actually had undiagnosed diabetes. A spam dataset where 5% of legitimate emails were marked spam because they had unusual characters. Wrong labels → wrong learning.

Distributional shift. The world the data came from is not the world the system is deployed in. A model trained on 2019 NEPSE prices does not know what 2025 NEPSE prices look like. A demand-forecasting model trained on pre-pandemic data does not know about pandemic shopping. The data was accurate when collected; the world moved.

Each of these has the same shape: the model is fine; the data-to-deployment gap is fatal. The job of a builder is to expect these gaps and design around them.

A worked Nepali example

Imagine you are building a small loan-default prediction model for a microfinance institution in Tarai. You have a CSV of past loans: borrower district, loan amount (in rupees), repayment history, default flag.

Five real things you would catch — only by looking at the data carefully — that would otherwise sink the model:

Amount is sometimes in lakhs. 4% of rows are entered as 5 (meaning Rs 5 lakh) instead of 500000. The model, looking at the column, would think those borrowers took out a Rs 5 loan and somehow defaulted. Catastrophic.
District names are spelled four ways. “Morang”, “morang”, “Morang District”, “मोरङ”. The model would treat these as four different districts and split its learning across them, never getting enough signal on any.
Some defaults are missing. Loans from 2024 that have not yet completed their term are marked as “not defaulted” — because they have not defaulted yet. The model would learn that recent loans never default, which is the opposite of true.
One officer’s loans default at twice the rate. Because that officer covers a single drought-prone district. The model would learn that this officer is the predictor, when really geography is.
The labels were collected only from completed loans. Loans that were written off without official default status (because the borrower vanished, say) are not in the dataset at all. The model has never seen a “disappear” outcome.

You will not catch any of these by feeding the data into a model and looking at accuracy. You will catch them by looking at the data. That is the central skill of this course.

What this course is, in one paragraph

Course 02 is the unglamorous twin of Course 01. In Course 01 you learned to call models and wire them into programs. In this course you learn the substrate — where data comes from (Chapter 2), how to clean what you find (Chapter 3), how to understand what you have (Chapter 4), how to prepare it for a model (Chapter 5), and how to do all of this without harming the people whose data you are touching (Chapter 6). The model itself does not appear until Course 03. That delay is the point.

Check your understanding

Quick check

—

A team in Lalitpur trains a loan-default model on three years of Kathmandu Valley microfinance data. The model performs beautifully in validation. They deploy it across Karnali Province and accuracy collapses. What is the most likely cause, in the language of this section?

The model is the wrong architecture; they should retrain with a deeper neural network.
Representation mismatch: the training data did not represent the population the system is now being used on. Karnali borrowers, livelihoods, and default patterns are not in the training data.
The model needs more compute at inference time.
Accuracy is unreliable in deployment; this is normal and should be ignored.

Quick check

—

You are reviewing a Nepali-Devanagari OCR dataset and notice that 200 of the 5,000 ground-truth labels are clearly wrong (a 'क' is labelled as 'ख', etc.). The model trained on this dataset shows 92% accuracy on the test set. What is most likely true?

92% accuracy is excellent; the 200 wrong labels are negligible and can be ignored.
The model has likely learned some of the wrong labels as 'correct' patterns, and the test-set accuracy is partially measuring how well it learned the noise. The real accuracy on clean data may be different in either direction.
The model should be retrained on a deeper network until it overcomes the label noise.
Label noise has no effect because modern models are robust to it.

What comes next

If you accept the premise — that data quality decides everything — the rest of this course is the toolkit. The next two sections of Chapter 1 cover what kinds of data you will encounter (numbers, text, images, time) and what a dataset’s life looks like from collection to model. Then Chapter 2 gets specific: where to actually go to fetch real data, and how to do it without breaking the rules of the place you fetch from.

Before you turn the page, one small homework: think of one dataset you have actually touched — your bank statement CSV, a class roster, a Google Sheet of student grades. Predict three things that are probably wrong with it. We will use that prediction as a calibration exercise in Chapter 3.