Features: turning reality into columns

A feature is a column the model can use

The word feature in this course means: a column of values the model will treat as input. Some features are obvious — loan_amount, borrower_age, district. Some are derived — months_since_last_loan, is_first_time_borrower, loan_to_income_ratio. The derived ones are often where the real signal is.

A simple loan dataset might arrive with columns like:

borrower_id, application_date, loan_amount, district, occupation, defaulted

Six columns. A model trained on these will work, in some basic sense. But the same dataset, after twenty minutes of feature engineering, might look like:

borrower_id, application_date, loan_amount, district, province, occupation,
occupation_category, application_month, application_dayofweek,
is_first_time_borrower, prev_loan_count, days_since_last_loan,
is_drought_year, defaulted

Fourteen columns. The model has much more to work with, and several of the new columns are exactly the patterns a human loan officer would use to assess risk.

The four sources of new features

Almost every useful feature you will ever engineer comes from one of four moves:

1. Decomposition. Break a complex value into simpler ones. A date becomes year, month, dayofweek. An address becomes district, province. A name becomes first_name, family_name.

df["application_date"] = pd.to_datetime(df["application_date"])
df["application_month"] = df["application_date"].dt.month
df["application_dayofweek"] = df["application_date"].dt.dayofweek
df["application_year"] = df["application_date"].dt.year

2. Combination. Two simpler columns become one richer one. A loan_amount and monthly_income become loan_to_income_ratio. A latitude and longitude become distance_to_kathmandu. A start_time and end_time become duration.

df["loan_to_income_ratio"] = df["loan_amount"] / df["monthly_income"]

3. Aggregation across history. Look at past rows for the same entity. For each borrower, count how many loans they have had before this one. For each district, compute the historical average default rate.

df["prev_loan_count"] = df.sort_values("application_date").groupby("borrower_id").cumcount()

4. External context. Bring in a column from outside the dataset. Was 2023 a drought year? Was the application made during festival season? What was the NEPSE index that week? Each of these turns into a column.

drought_years = {2014, 2017, 2023}
df["is_drought_year"] = df["application_year"].isin(drought_years).astype(int)

Most of the work of feature engineering is some combination of these four moves. Once you have the four in your head, you start to see them everywhere.

A worked example: turning a loan row into features

Starting from a single raw row:

{
  "borrower_id": "B-12847",
  "application_date": "2024-08-15",
  "loan_amount": 50000,
  "monthly_income": 18000,
  "district": "Morang",
  "occupation": "Tea stall owner",
  "defaulted": True,
}

After feature engineering, the same row is enriched to:

{
  "borrower_id": "B-12847",
  "loan_amount": 50000,
  "log_loan_amount": 10.82,                 # log-transformed for skewness
  "monthly_income": 18000,
  "loan_to_income_ratio": 2.78,             # combination feature
  "district": "Morang",
  "province": "Province 1",                 # decomposition + lookup
  "occupation": "Tea stall owner",
  "occupation_category": "small_business",  # mapping to a coarser category
  "application_month": 8,                   # decomposition
  "application_dayofweek": 3,               # decomposition (Thursday)
  "application_quarter": 3,                 # decomposition
  "is_festival_season": False,              # external context (Dashain/Tihar months)
  "is_drought_year": True,                  # external context
  "prev_loan_count": 0,                     # aggregation across history
  "is_first_time_borrower": True,           # aggregation across history
  "defaulted": True,
}

Sixteen features instead of six. Each is computable from the raw data plus a small amount of external context. The model now has much more to work with — and importantly, the features encode the kind of judgments a thoughtful loan officer would already be making.

Three traps to avoid

Trap 1: Including the target. A feature derived from the outcome you are trying to predict is called leakage, and it will make your model look magical in training and fail completely in production. A column “did this loan get rolled over after default” is downstream of “did this loan default” — including it as a feature lets the model cheat. Always ask: “could this value have been known at the time the prediction needs to be made?” If not, leave it out.

Trap 2: Future information. Time-series datasets are full of this. Aggregating “average default rate in this district” using all data including future is leakage; aggregating using only data up to each row’s date is legitimate. The pandas expanding() and rolling() functions help you do this correctly.

Trap 3: One-hot exploding. A categorical column with thousands of unique values (every borrower_id, every street address), when naïvely encoded, produces thousands of new columns and a model that overfits. We discuss encoding in the next section, but the principle is: high-cardinality categoricals need thought, not blind encoding.

A practical workflow

For a new dataset, a feature-engineering pass typically looks like:

1. Decomposition first. Dates → date parts. Addresses → districts. Easy, mechanical, almost always useful.
2. Domain combinations next. Talk to someone who knows the domain. What ratios, differences, durations would they look at if they were making this decision? Build those columns.
3. History aggregation. For each entity (borrower, customer, school), count and aggregate their past. Be careful about the time-order trap.
4. External context last. Bring in calendars, holidays, weather, exchange rates as needed.

Save the engineered DataFrame and document each new column in your data/README.md — what it means, how it was computed, and what could go wrong with it. Future-you will need to remember.

Check your understanding

Quick check

—

A teammate adds a feature called `was_loan_renewed_after_default` to a loan-default prediction model. The training accuracy jumps from 78% to 99%. What is the most likely cause and the right response?

1. AThe feature is extremely informative; ship the model.
2. BThis is target leakage. `was_loan_renewed_after_default` could only be known *after* the default has happened, so including it lets the model cheat in training. In production, this column would not be available at prediction time, and accuracy would collapse. Remove the feature, then look at the remaining features for honest signal.
3. CIncrease model regularisation to bring training accuracy down.
4. DThe model is finally working correctly; deploy it.

Check answer→

Quick check

—

You are predicting Nepali school dropout rates from administrative data. Your raw data has student ages, attendance counts, exam scores, and home address. Which of the following is the most useful *decomposition* feature to engineer?

1. AConcatenate all four columns into one string.
2. BDecompose 'home address' into 'district' and 'province'. The raw address text is high-cardinality and hard for a model to use, but district and province are well-behaved categorical features with meaningful explanatory power for dropout (different districts have very different schooling cultures, transport availability, and economic conditions).
3. CConcatenate the four columns into a single hashed integer.
4. DUse raw exam scores divided by attendance count as the only feature.

Check answer→

What comes next

You can identify what features to engineer. The next section is about how to make those features model-ready: turning categorical text into numbers (encoding) and putting numeric features on comparable scales (scaling). Both are mechanical but important — and both have specific Nepali quirks worth knowing.