ailiteracynepal 🇳🇵
Text size

Chapter 03 · Section II · 28 min read

pandas: loading and exploring tables

pandas takes the list-of-dictionaries shape from Chapter 2 and gives it a fast engine and a friendly grammar. It is how almost every working data person in Python touches tables.

In Chapter 2 you read a CSV into a list of dictionaries, looped over it, and totalled a column. That worked, and it was honest work — you saw every step. Now imagine doing the same thing on a year of NEPSE data with twelve columns. The loop version becomes tedious to write and slow to run. pandas is the library that does the same thing in one or two lines, faster, and with less to type.

Installing pandas

In your terminal, from the project folder:

pip install pandas

In a notebook:

!pip install pandas

The convention is to import it as pd:

import pandas as pd
print(pd.__version__)

Reading a CSV in one line

Recall the data/prices.csv from Chapter 2:

date,close
2026-06-23,2105.4
2026-06-24,2128.9
2026-06-25,2117.3
2026-06-26,2134.7

In Chapter 2 you wrote half a screen of code to read this and convert the prices to floats. In pandas:

import pandas as pd

prices = pd.read_csv("data/prices.csv")
print(prices)

That is it. pd.read_csv reads the file, takes the first row as column names, and figures out the data types of each column automatically. The result — prices — is a DataFrame, pandas’s table type.

Print it:

         date   close
0  2026-06-23  2105.4
1  2026-06-24  2128.9
2  2026-06-25  2117.3
3  2026-06-26  2134.7

The leftmost column with 0, 1, 2, 3 is the index — a positional label for each row. By default it is just 0, 1, 2, ...; later you can set a date column as the index, which is often what you want for time series.

The four functions to know in your first hour

Almost every exploration of a new dataset starts with these four:

prices.head()        # first 5 rows
prices.tail()        # last 5 rows
prices.shape         # (rows, columns) — e.g. (4, 2)
prices.describe()    # summary statistics of numeric columns

describe() gives count, mean, std (standard deviation), min, max, and the quartiles for each numeric column. It is the fastest way to get a feel for a dataset you have never seen before.

Two more, for column-level inspection:

prices.columns       # Index(['date', 'close'], ...) — the column names
prices.dtypes        # data types of each column

Working with columns

A column of a DataFrame is called a Series. You access it by name:

prices["close"]            # the close column as a Series
prices["close"].mean()     # 2121.575
prices["close"].max()      # 2134.7
prices["close"] * 1.05     # every price multiplied by 1.05

A Series behaves a lot like a 1D NumPy array — most of the operations from the last section work on it. The difference is that a Series has an index (the labels) attached.

You can also add new columns by assignment:

prices["close_npr_lakh"] = prices["close"] / 100000

This adds a new column derived from an existing one. Every row of the new column is computed at once — no loop required. You are using the vectorisation idea from NumPy without having to think about it.

Filtering rows

The pandas version of the boolean-mask idea from NumPy:

prices[prices["close"] > 2120]

That returns a new DataFrame containing only the rows where the close price was above 2120. Two days, in our small example. The same pattern works for any condition:

prices[prices["date"] >= "2026-06-25"]
prices[(prices["close"] > 2100) & (prices["close"] < 2125)]

Note: combine conditions with & and | (not and and or), and wrap each condition in parentheses. This is one of the small papercut habits of pandas — get burnt by it once and you remember forever.

groupby: the single most useful operation

If there is one pandas operation you will use every week for the rest of your life, it is groupby. Imagine a transactions table:

import pandas as pd

transactions = pd.DataFrame([
    {"vendor": "Khalti",  "amount": 5000, "date": "2026-06-25"},
    {"vendor": "eSewa",   "amount": 1200, "date": "2026-06-25"},
    {"vendor": "Khalti",  "amount":  450, "date": "2026-06-26"},
    {"vendor": "FonePay", "amount":  900, "date": "2026-06-26"},
])

To get the total amount per vendor:

transactions.groupby("vendor")["amount"].sum()

Result:

vendor
Khalti    5450
eSewa     1200
FonePay    900
Name: amount, dtype: int64

Read the chain left to right: take the table, split it into groups by the value of vendor, then within each group take the amount column and sum it. Three lines of intent expressed in a single line of code. The Chapter 2 version of this (a function with an if inside a loop) was twelve lines. The pandas version is one.

You can group by multiple columns, and apply other summaries (mean(), count(), max()):

transactions.groupby(["vendor", "date"])["amount"].sum()
transactions.groupby("vendor")["amount"].mean()
transactions.groupby("vendor")["amount"].count()

Saving a DataFrame back to disk

You will almost always want to write a result somewhere. pandas matches its readers with writers:

prices.to_csv("data/prices_out.csv", index=False)
prices.to_json("data/prices_out.json", orient="records", indent=2)

index=False keeps the implicit 0, 1, 2, ... index from being written as a stray column. orient="records" writes JSON as a list of dictionaries (the shape from Chapter 2), which is almost always what you want.

Check your understanding

Quick check

—

A DataFrame `sales` has columns 'district' and 'amount'. Which one-liner gives total sales per district?

Quick check

—

You wrote `prices[prices['close'] > 2100 and prices['close'] < 2125]` and pandas raised a ValueError about the truth value of an array. What is the fix?

What comes next

Numbers in a DataFrame are useful, but humans read pictures faster than tables. In the next section we draw the prices we have been computing — a line chart, a bar chart, a scatter plot — using matplotlib. Once you can see your data, half the bugs in a real dataset announce themselves on sight.