Neural network basics

A neuron is a function

Imagine we want a tiny model that estimates how likely a Khalti transaction is to be fraud, based on three features:

• amount — the size of the transaction in rupees.
• time_of_day — encoded as 0 (3am) to 1 (3pm).
• new_device — 1 if this device has not been seen before for this user, 0 if it has.

A single neuron will produce a score between 0 and 1 — the higher the score, the more the neuron thinks “fraud.”

The neuron does the following:

score = squish(w₁ × amount + w₂ × time_of_day + w₃ × new_device + b)

That is the entire computation. Each input is multiplied by a weight (w₁, w₂, w₃). The results are added, plus a constant bias b. Finally, the sum is passed through an activation function — the “squish” — which turns any real number into a value between 0 and 1.

A common squish for this kind of output is the sigmoid: very large positives become close to 1; very large negatives become close to 0; zero becomes 0.5. The activation function is what lets a neuron behave non-linearly. Without it, a network of neurons would be no more powerful than linear regression.

What the weights mean

After training on millions of historical transactions, the neuron settles on weights that capture useful patterns. Suppose, for our fraud neuron:

• w₁ = 0.0001 (per rupee of transaction size)
• w₂ = −1.2 (per unit of time, so daytime brings the score down)
• w₃ = 2.0 (new device is a strong fraud signal)
• b = −0.5 (a small bias toward “not fraud”)

Now a transaction comes in: Rs. 50,000, at 4am (time_of_day = 0.1), from a new device.

raw = 0.0001 × 50,000 + (−1.2) × 0.1 + 2.0 × 1 + (−0.5) raw = 5 − 0.12 + 2 − 0.5 = 6.38

That is a fairly large positive number. After squishing through sigmoid (which maps large positives close to 1), the neuron returns a score of about 0.998. The neuron is calling this fraud with high confidence.

If the same transaction had come from a familiar device (new_device = 0) at 3pm (time_of_day = 1), the raw value would be:

raw = 5 + (−1.2) − 0 + (−0.5) = 3.3

Still positive (it is a big transaction), but smaller. After sigmoid, the score is about 0.96. Still suspicious, but less.

A single neuron has learned — through the choice of weights — to combine multiple features into a single probability-like score. This is the whole job of a neuron.

Why one neuron isn’t enough

The fraud neuron above is doing linear-then-squished. The “linear” part is the same as linear regression. The “squished” part adds a single non-linearity at the end.

This is good for the simplest classification, but it has a hard ceiling. A single neuron can only learn linearly separable patterns — patterns where you could draw a single straight line in feature space that puts the frauds on one side and the not-frauds on the other.

Many real patterns aren’t linearly separable. Fraud at very small amounts at 3am from familiar devices is a different pattern from fraud at huge amounts at noon from new devices. A single neuron cannot represent two distinct patterns at once.

The fix is to use many neurons, in layers. The output of one layer becomes the input of the next. Each layer can extract a slightly higher-level pattern from the layer before. With enough layers, the network can represent virtually any function we have data for.

That is the core idea of a neural network — and of deep learning when the number of layers is large.

A miniature network

Suppose instead of one neuron, we use three neurons in a hidden layer, each looking at the same three input features but with different weights. Each neuron has learned to detect a different “kind” of fraud:

• Neuron 1: high score when amount is big.
• Neuron 2: high score when new device + odd time.
• Neuron 3: high score when amount is small but combined with many transactions in a short window.

Their three outputs become inputs to a final neuron that combines them — also with learned weights — to produce the final fraud probability.

The “training” is just finding all these weights at once, across all four neurons, such that the network’s final outputs match the labelled training data. The number of weights gets large quickly, but the algorithm is the same as it was for linear regression: define an error on training data, find weights that reduce it.

What “training” actually does

In our linear-regression chapter, we mentioned gradient descent — adjusting weights in the direction that reduces error, one tiny step at a time. The same algorithm trains every neural network you have ever heard of. The only twist is that, in a network, the error has to be sent backwards through the layers to figure out how each weight contributed to it. That backwards flow is called backpropagation — the topic of the next section.

What you should hold in your head, before that next section, is the basic shape of a neuron and the basic shape of a network. Everything bigger — ChatGPT’s 175 billion parameters, Gemini’s image generation, your phone’s OCR — is the same shape with vastly more neurons and vastly more arithmetic.

Why this matters for Nepal

A surprising number of practical Nepali AI problems can be solved with very small neural networks — networks with a few hundred neurons, trained in minutes on a laptop. Devanagari character classification, sentiment on customer feedback, predicting whether a credit application is likely to default. The mythology around AI being a “giant model” question is mostly wrong. The right model is the smallest one that does the job.

The hype, of course, is the other way around. The press wants stories about billion-parameter models. The reality, in working Nepali AI shops, is mostly small networks on small datasets, doing small useful things very well. Knowing the basic shape of a neuron is enough to read the latter accurately, and to recognise when someone is overselling the former.

Check your understanding

Quick check

—

A single artificial neuron in a neural network does which of the following?

1. AStores the entire training dataset in memory
2. BTakes a weighted sum of its inputs and passes the result through an activation function
3. CGenerates random outputs unrelated to the inputs
4. DTranslates text from one language to another all by itself

Check answer→

What comes next

We have a neuron and a tiny network. The next section is about how the network finds its weights — the backpropagation algorithm — and the layer types that turn a basic network into the architectures behind modern AI.