What "training a model" actually does · Machine Learning, Hands-On

You have used models other people trained — Claude in Course 01, Tesseract in Course 01, the models you read about throughout Learning AI. This course is the first time you train your own. Before any code, a short, plain explanation of what training a model actually is. It is far less mysterious than the news headlines make it sound, and understanding it makes scikit-learn — the library we’ll meet in the next section — feel like an old friend.

A model is a function with adjustable knobs

A model, in machine learning, is a mathematical function. You give it inputs (your features from Course 02), it produces an output (a prediction). What makes a model a model — and not just any function — is that it has adjustable knobs inside it, called parameters.

Take the simplest possible model. To predict whether a microfinance loan will default, we’ll start with just one feature — loan_to_income_ratio — and decide:

If ratio > threshold, predict default. Otherwise predict no-default.

That model has one knob: the threshold. Different threshold values produce different predictions. Some thresholds will be good (the model agrees with reality often). Others will be bad. Training the model is finding the threshold that produces predictions which agree with reality as often as possible.

That, fundamentally, is what every machine learning algorithm does. The models get bigger. The number of knobs grows from one to ten to ten million. The math of how to adjust them gets cleverer. But the shape is unchanged: a function with knobs, and a procedure for setting those knobs by looking at examples.

The three pieces every training procedure has

Every machine learning algorithm — from a one-knob threshold model to GPT-4 — has three pieces:

The model itself. The function with knobs. For a threshold model, one knob. For linear regression, one knob per feature plus an offset. For a neural network, millions of knobs arranged in layers.
A loss function. A number that measures how wrong the model is on a piece of data. Lower is better. A loss of zero means the model is perfectly right on every training example. (Spoiler: this is almost never achievable, and would be suspicious if it were.)
An optimisation procedure. A recipe for adjusting the knobs so the loss goes down. The simplest version: try lots of values, keep the one with the lowest loss. Real algorithms are much faster but morally identical.

You will rarely have to write any of these three yourself. scikit-learn — and PyTorch, TensorFlow, etc — provide models, losses, and optimisers as off-the-shelf parts. Your job is to assemble them.

A picture, in two minutes

Imagine plotting loan_to_income_ratio on the x-axis and defaulted (0 or 1) on the y-axis for 1,000 past loans. You will see two clouds — non-defaulters mostly on the left (lower ratios), defaulters mostly on the right (higher ratios), with some overlap in the middle.

Training a threshold model is drawing a vertical line on this plot and asking: “If I predict ‘default’ to the right of this line and ‘no-default’ to the left, how many predictions agree with the actual labels?” You try different vertical-line positions. You pick the one with the most agreements.

That is the simplest possible model and the simplest possible training. Everything in this course is a more powerful version of the same idea.

What the training data is, and what the model becomes

Two terms we will use constantly:

Training data is the set of past examples you show the model — each row a feature vector (loan amount, ratio, district, etc.) and a label (defaulted or not). The training process consumes these examples to adjust the model’s knobs.

A trained model is the model with its knobs set to the values the training procedure landed on. You save the trained model to a file. You load it later and use it to predict on new loans the model has never seen.

This split — train once, predict many times — is the heart of how machine learning is deployed in production. A microfinance institution does not retrain the model on every new application. They train it once a week (or month), save it, and use the saved model for thousands of new applications until the next retrain.

Two big families: classification and regression

Almost every supervised ML problem (we’ll define “supervised” properly in the next section) falls into one of two families based on what kind of label you’re predicting:

Classification predicts a category. Will this loan default? Yes/no. Is this SMS spam? Yes/no. Which of 8 disease categories does this X-ray show? The output is one of a finite set of labels. Most of Chapter 2 is about classifiers.

Regression predicts a number. How much will this Kathmandu flat rent for? In NPR. What will NEPSE close at tomorrow? In index points. How many days until this customer next visits? The output is a continuous numeric value. Chapter 3 covers regression.

The distinction matters because the loss functions, evaluation metrics, and even some of the models are different. But the shape — function with knobs, loss, optimiser — is identical.

A small reframing of what we did in Course 02

The whole arc of Course 02 was preparing data so that a model could train on it well. With this section’s vocabulary, you can now name what each Course 02 chapter was doing:

Chapter 1-2 (getting data) → creating the training set.
Chapter 3 (cleaning) → removing noise that would corrupt the loss.
Chapter 4 (understanding) → checking that the training set actually represents the population you want to predict on.
Chapter 5 (preparing) → engineering features the model can use, scaling them, splitting into train/val/test so you can train and evaluate honestly.
Chapter 6 (responsibility) → noticing what bias the training data carries before the model amplifies it.

Every one of those chapters was setting you up for the moment in the next section, when you write model.fit(X_train, y_train) for the first time, and a real model trains on real data on your laptop.

Check your understanding

Quick check

—

In the language of this section, what is the most accurate description of what happens when you 'train a machine learning model'?

The model becomes conscious of the data and starts making intelligent decisions.
An optimisation procedure adjusts the model's internal parameters (knobs) so that its predictions on training data agree with the training labels as well as possible, measured by a loss function.
The model copies the training data so it can recall the answers later.
Training requires no math — modern frameworks handle everything automatically.

Quick check

—

You are predicting tomorrow's NEPSE closing index value (a number, e.g. 2142.7). Which family of ML problem is this, and why does the distinction matter?

Classification, because the index has finitely many possible values in practice.
Regression, because the output is a continuous numeric value. The distinction matters because regression uses different loss functions (typically squared error) and different evaluation metrics (RMSE, MAE) from classification — so you reach for different scikit-learn classes and different scoring code.
Unsupervised learning, because we don't know tomorrow's value.
Both classification and regression simultaneously.

What comes next

You know what training is doing. The next section meets the library that does it for you — scikit-learn, the workhorse of the Python ML ecosystem. By the end of Section 2 you’ll have a clean mental model of how scikit-learn’s .fit() and .predict() correspond to the three pieces (model, loss, optimiser) we just named. Then Section 3 puts the whole thing together: your first end-to-end classifier, training on real data, predicting on data it hasn’t seen.