scikit-learn: the maker's ML library · Machine Learning, Hands-On

If there is one Python library every working ML practitioner has open every day, it is scikit-learn. It is twenty years old, written and maintained by some of the best statisticians and software engineers in the field, free, and free of cleverness — it does what you’d expect, the way you’d expect, every time. By the end of this section you will have it installed, you will know the universal shape of every scikit-learn model, and you will be ready to train your first real classifier in Section 3.

Installing scikit-learn

In your activated virtual environment from Course 01 Chapter 4:

pip install scikit-learn

The convention is to import it as sklearn:

import sklearn
print(sklearn.__version__)

You will rarely import the whole library. Instead, you import the specific pieces you need — a pattern we’ll see in every example.

The four imports you’ll do all the time

Almost every scikit-learn script reaches for four things:

# A model — the function with knobs
from sklearn.linear_model import LogisticRegression

# Preprocessing — same library, no separate install
from sklearn.preprocessing import StandardScaler

# A scoring function — to judge how well the model did
from sklearn.metrics import accuracy_score

# Splitting — to honestly evaluate
from sklearn.model_selection import train_test_split

You already saw train_test_split and StandardScaler in Course 02. LogisticRegression is one of dozens of available models. accuracy_score is one of many available metrics. The library is huge but well-organised, and the four-import pattern covers about 80% of working code.

The universal API: `.fit()` and `.predict()`

Here is the single most important fact about scikit-learn: every model has the same interface. Whether you use a one-knob threshold classifier or a thousand-knob random forest, the code looks like this:

model = SomeModelClass(...config...)  # create the model
model.fit(X_train, y_train)            # train it
predictions = model.predict(X_test)    # predict on new data

Three lines, always. The model class changes, the config arguments change, the data changes — the shape never changes. This consistency is why scikit-learn is the workhorse: once you’ve used it for one model, you’ve used it for all of them.

A more complete example, end to end:

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# X is your feature DataFrame; y is your target Series — from Course 02
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y,
)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

predictions = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, predictions))

Ten real lines. They train a real model on real data and tell you, honestly, how well it did. The next section walks through this code line by line with the loan-default dataset from Course 02. For now, sit with the shape.

What `fit` does — connecting to the previous section

In Section 1 we said training a model is: an optimisation procedure adjusts the knobs to minimise a loss function. model.fit(X_train, y_train) is that optimisation procedure, hidden behind one line of Python.

For LogisticRegression, the loss function is something called the logistic loss (or “log-loss”), and the optimiser is one of a few standard algorithms (LBFGS by default). You did not have to name any of these. scikit-learn made the sensible choice. When .fit(...) returns, the model’s internal knobs are set to the values that minimised the loss on your training data.

To peek at the knobs after training:

print(model.coef_)       # the weights (one per feature)
print(model.intercept_)  # the offset

For a model with five features, coef_ is an array of five numbers — the model’s idea of how each feature contributes to the prediction. Reading these is a great way to sanity-check what the model learned: do the weights have the sign you’d expect (loan_to_income_ratio should have a positive weight for predicting default), and roughly the magnitude you’d expect?

What `predict` does

model.predict(X_test) applies the trained model to each row of X_test and returns the predicted label. For a classifier, the result is an array of class labels (e.g. [0, 1, 0, 0, 1, ...]). For a regressor, an array of numbers.

A close cousin is predict_proba, available on most classifiers:

probabilities = model.predict_proba(X_test)
print(probabilities[:5])

This returns the probabilities the model assigns to each class, not just the final label. Useful when you want a “confidence” score — and crucial when you want to adjust the decision threshold (e.g. only flag a loan as risky if predicted probability > 0.7, not just > 0.5).

A small map of what’s in the library

A short, non-exhaustive tour:

sklearn.linear_model — linear/logistic regression and friends. Simple, fast, interpretable.
sklearn.tree — decision trees. We use these in Chapter 2.
sklearn.ensemble — random forests, gradient boosting. Almost always strong out of the box. We’ll meet these in Chapter 5.
sklearn.neighbors — k-nearest neighbours. Surprisingly competitive for small datasets.
sklearn.svm — support vector machines. Powerful, sometimes finicky to tune.
sklearn.preprocessing — scalers, encoders, polynomial features.
sklearn.model_selection — splitting, cross-validation, hyperparameter search.
sklearn.metrics — every evaluation metric you’ll ever want.
sklearn.pipeline — the Pipeline class from Course 02 Chapter 5.

The official docs (scikit-learn.org) are exceptionally good — every class has examples, every argument has a description, and the cross-references are sane. Read them often.

A practical note on warnings

Run any scikit-learn script and you may see warnings — about convergence, about feature names, about deprecated arguments. Read them. Most are useful. The most common one you’ll hit early:

ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

This means the optimiser ran out of iterations before finding the bottom of the loss. Two common fixes: pass max_iter=1000 (or 5000) to give it more iterations, or scale your features first with StandardScaler so the optimiser has an easier job. We did both in the example above.

The other one you’ll see:

UserWarning: X does not have valid feature names...

This appears when you pass a NumPy array to .fit() but a pandas DataFrame to .predict() (or vice versa). Stick to one and the warning goes away.

Check your understanding

Quick check

—

A friend wants to switch from a linear regression to a random forest for a Kathmandu rent prediction project. They worry about how much code they'll need to rewrite. What does this section suggest?

Switching models in scikit-learn requires a full rewrite — most of the code will change.
Almost no code needs to change. Because every scikit-learn model follows the same fit/predict interface, swapping `LinearRegression()` for `RandomForestRegressor()` typically requires changing only the import and the model-instantiation line. The data loading, splitting, scaling, training call, and evaluation code stay identical. This consistency is the central reason scikit-learn became the standard.
You cannot switch from linear regression to random forests in scikit-learn — they are in incompatible libraries.
You need to rewrite the training loop manually for each model.