Presenting results honestly

The five things a result needs

A working report contains five elements, in order. Skip any one and the report becomes either unconvincing or misleading.

1. The framing recap. Who is this for? What decision does it inform?
2. The headline metric and its baseline. One trustworthy number.
3. The per-group breakdown. Where does the model work well? Where doesn’t it?
4. The limitations. What does the model not know? When should it not be used?
5. The next steps. What would you do with another week, another month?

That is the whole report. It fits on one page. The rest is supporting detail.

A worked report — NEPSE direction predictor

What an honest, useful one-pager looks like:

# NEPSE Direction Predictor — Model Report v1.0
Author: Your Name
Date: 2026-06-29

## Framing

This model is built for a small retail investor who wants a directional
hint on whether tomorrow's NEPSE will be up or down. It is not a buy
signal. It is one input among many. The framing document is in
`docs/framing.md`.

## Results

- 5-fold time-series CV F1: 0.62 (± 0.03)
- Test set F1 (held out, last 250 days): 0.49
- Baseline (always predict up): F1 = 0.61
- Sample size: 1,243 trading days, of which 250 in test.

The gap between CV and test F1 reflects market-regime shift in 2026
(see "Limitations"). The model marginally beats baseline on the CV
period but underperforms on the test set — likely the dominant signal
is regime change rather than model weakness.

## Per-period performance

20-day rolling F1 on the test set:
- Q1 2026: 0.58 (markets steady)
- Q2 2026: 0.41 (high volatility, policy news)

The model is most reliable in steady markets and least reliable during
high-volatility news-driven days — exactly the days where directional
prediction matters most. This is the central caveat for any user.

## Limitations

1. The model has seen 5 years of data. The training period does not
   include comparable events to those in Q2 2026. Predictions during
   regime shifts should not be trusted.

2. The model uses only price-based features (returns, volatility,
   moving averages). It has no information about news flow, policy
   announcements, or macroeconomic indicators — all of which drive
   short-term direction.

3. The model is *directional* — it does not predict magnitude. A
   "predicted up" day can still see a 0.1% gain or a 4% gain.

4. The model will deteriorate as the market evolves. It should be
   retrained at least monthly, ideally weekly.

## Next steps

If we had another week:
- Add news-sentiment features from public Nepali financial news.
- Add a meta-model that predicts when the base model should be trusted
  (calibration / confidence model).
- Build a small daily-update pipeline that retrains and publishes the
  prediction at 6 AM with the prior day's close baked in.

If we had another month:
- Replace point predictions with calibrated probabilities and
  explicit uncertainty bands.
- A/B test the directional hint UX with real users to measure whether
  it helps decisions.

That is the whole document. Stakeholders read it. They make decisions on it. It is honest about what works and what doesn’t.

The charts that earn trust

Three charts pay for themselves in every report.

Chart 1 — actual vs predicted (regression) OR confusion matrix (classification)

For regression: scatter plot of actual y vs predicted y, with a diagonal y=x reference line. Lets the reader see where the model is biased.

For classification: heatmap of the 2×2 (or N×N) confusion matrix, labelled with class names. Lets the reader see which mistakes the model is making.

Chart 2 — performance over time / per group

For time-series models: rolling F1 (or MAE) over the test set, with a 20-day window. Lets the reader see when performance is consistent and when it isn’t.

For grouped problems: bar chart of per-group F1 (or MAE) with sample size annotations. Lets the reader see which groups are well-served and which aren’t.

Chart 3 — calibration (for probabilistic classifiers)

For any classifier that outputs probabilities: a reliability diagram showing how well the predicted probability matches the actual frequency. A model that says “70% chance of up” should be right 70% of the time on those days. If the actual is 55%, the model is overconfident — which matters when a user is making decisions based on the probability.

from sklearn.calibration import calibration_curve
import matplotlib.pyplot as plt

probs = model.predict_proba(X_test)[:, 1]
prob_true, prob_pred = calibration_curve(y_test, probs, n_bins=10)

plt.plot(prob_pred, prob_true, marker="o")
plt.plot([0, 1], [0, 1], "k--", alpha=0.5)
plt.xlabel("Predicted probability")
plt.ylabel("Actual probability")
plt.title("Calibration")
plt.show()

Reading the calibration plot is the single fastest way to spot an overconfident or underconfident model. It belongs in every probabilistic report.

Writing for non-technical readers

The hardest skill in ML reporting is writing for readers who don’t know what F1 is. A short translation guide:

• “F1 of 0.62” → “Of the days the model predicts up, 60% are actually up, and the model catches about 65% of the days that are actually up.”
• “MAE of 4,200 NPR” → “Typical prediction error is around Rs 4,200 — predictions are usually within Rs 4,000 of the true rent.”
• “AUC of 0.78” → “If we pick a random day where the index went up and a random day where it went down, the model would assign higher probability to the up day about 78% of the time.”

When in doubt, use the plain-language version. Add the technical name in parentheses for the technical reader.

What honest reports do for your career

Three real benefits — none of them visible from inside the work:

1. Stakeholders trust you. When you lead with limitations, the listener relaxes — they know you’re not going to hide bad news. The first hard project where this earns you trust will pay off for years.

2. You catch your own mistakes. Writing “the model deteriorates in high-volatility periods” forces you to check whether it does. Often you discover an even worse failure mode in the process.

3. The report becomes your portfolio. A clean one-pager with framing, results, limitations, and a chart is more compelling to a hiring manager than a 50-line snippet of impressive accuracy. It shows judgment.

Closing Course 03

This is the end of Course 03. You started the course as someone who could call models other people trained. You leave as someone who:

• Trains their own classifiers and regressors with scikit-learn.
• Evaluates honestly with precision, recall, confusion matrices, and cross-validation.
• Recognises and fixes overfitting.
• Tunes hyperparameters without leaking test data.
• Frames real problems for users with names and decisions.
• Reports results that earn trust, not just admiration.

These six skills, in combination, are the working competence of a junior-to-mid ML practitioner. Most of the world does not have them. You do.

Course 04 begins — Building with Language Models

Classical ML works beautifully on tabular data with engineered features. It is not the modern way to handle language, conversation, summarisation, or open-ended generation. For that, you need large language models — and that is what Course 04, Building with Language Models, is about.

You’ll learn the modern maker’s path: prompting in code, getting structured output, giving models memory across turns, semantic search with embeddings, retrieval-augmented generation, and the careful construction of an AI assistant for a narrow, real-world Nepali use case. The shape will be familiar — fetch, prepare, call, use — but the model is now ChatGPT-class, the toolkit is now anthropic or openai, and the scope of what you can build expands enormously.

The substrate Course 02 and Course 03 built — clean data, honest evaluation, careful framing — applies to LLM-based projects exactly as it does to classical ML. Course 04 leverages everything you already know.

When you’re ready, turn the page.