Bias you would see in Nepal — caste, gender, language, region · AI, Ethics, and Society

Most of what is written about algorithmic bias is written in American. The case studies are American — COMPAS in Florida courts, Amazon’s hiring tool, facial recognition at Detroit precincts. The protected categories are American — Black, Hispanic, “low-income ZIP.” If you read enough of this work, you start to imagine bias as something that arrives in those shapes, and you start to miss the shapes it actually takes here. Nepal’s bias has a different grammar. The variables are different, the proxies are different, the regions are different, and the institutions that produce the biased data are different. Borrowing the American checklist will catch some of what matters and miss most of it. You have to learn the local grammar.

Caste, and the surname problem

The single most important thing any team building a screening model in Nepal needs to understand is that surnames carry caste. A model that uses an applicant’s name as a feature — or any feature derived from it, including email address, social media handle, or matched school yearbook — is, in effect, using caste as a feature. A model that has seen enough surnames during training will have learned the association even if no one told it explicitly. Sharma, Adhikari, Bhattarai, Pandey on one axis; Bishwakarma, Pariyar, Damai, Sarki on another; Yadav, Mahato, Sah, Mandal on a third. The model does not know what caste is. It knows what names are correlated with what outcomes, and outcomes in Nepal have been shaped by caste for centuries.

This is not theoretical. A hiring screen trained on the last five years of hires at a Kathmandu tech firm will learn, without anyone designing it to, that some surnames are over-represented in offers and others are under-represented in interviews. It will then “predict” the same pattern forward, and the team will report that the model is performing at 92% accuracy against the historical baseline — which is precisely the problem. The baseline is the bias. Beating it is reproducing it.

Gender, and the Bagmati pipeline

Nepali tech, like Nepali tech everywhere, has a gender problem. Women are 15-20% of the workforce in most Kathmandu software firms, less in senior roles, and the data shows it. A model trained on “successful employees” at one of these firms will, by construction, be a model of “successful male employees in Bagmati who joined between 2018 and 2024.” Deploy it to screen the next year’s applicants and it will quietly down-rank women, not because anyone wrote a rule, but because the historical pattern is the rule the model learned.

The same problem appears in credit data, in tenancy data, in startup-funding records, in anything where men have historically been the named actor. A microfinance model trained on cooperative records from the early 2000s will have most loans recorded against male heads of household, even when the actual borrower and repayer was a woman in the group. The model learns that female-named accounts are unusual and treats unusual as suspicious. The bias is upstream, in the recording practice, but the harm shows up downstream in who gets approved.

Language, and the Maithili gap

Large language models read the internet, and the internet in Nepal is not in Nepali in any uniform way. It is in Khas-Nepali heavily, in English heavily, in romanised mixed scripts on chat apps, and almost nothing in Maithili, Tamang, Limbu, Tharu, Magar, Newar, Bhojpuri, Awadhi, or Doteli. A model that “speaks Nepali” almost always means a model that speaks Khas-Nepali — and even then, mostly Kathmandu-register Khas-Nepali.

The downstream effects are real. A translation model will mangle Maithili honorifics or refuse to translate them at all. A speech-recognition system trained on Kathmandu accents will hear a Tarai speaker as if through static. A school assessment tool that scores written essays — especially one trained on English first and adapted to Nepali second — will systematically rate Western-style argumentative essays higher than the citation-heavy, more deferential style that Nepali secondary education actually teaches. None of these models was designed to be unfair. All of them are, and predictably so, because the training distribution did not include the populations they are now being asked to serve.

Region, and the two Nepals

Almost every dataset in Nepal that a model could be trained on was generated in a handful of districts. The bank’s records are mostly Kathmandu, Lalitpur, Bhaktapur, Kaski, Chitwan, Morang. The telecom usage data is biased toward urban towers. The hospital records exist in Kathmandu’s tertiary hospitals in detail, in district headquarters in summary, and barely anywhere else. Even the “national” datasets — household surveys, education statistics — under-sample the most remote wards.

A fraud-detection model trained on this data learns what “normal” transactions look like, and “normal” turns out to mean what people in the Valley do. A Madhesh trader sending many small repeated transfers across the Indian border in a remittance corridor will look anomalous to the model — not because the trader is doing anything wrong, but because the trader’s behaviour was under-represented in the training set. The model flags more Madhesh transactions, the bank investigates more Madhesh customers, those customers experience worse service, and the next iteration of the data records “more flags in Madhesh” as a fact. The feedback loop tightens.

The cleanest name for this is the two-Nepals problem: models are trained on the urban, connected, Khas-Arya, Bagmati-weighted Nepal, and then deployed across a country that is mostly none of those things. The training distribution and the deployment distribution are different countries. The model performs well on the first and unfairly on the second, and the people running it are usually closer to the first.

What an actual mitigation looks like

Once a team accepts that a model is producing biased outcomes, the wrong responses are loud and the right responses are slow. The wrong responses include: ignore it, the numbers are still good; rebrand the model and ship anyway; add a disclaimer and call it deployed; blame the users for being unusual. None of these change what the model does to people.

The right responses are concrete and unglamorous. Audit by group — split your test data by caste-cluster, by gender, by province, by language, and report the model’s error rates and acceptance rates for each. Change the training data — collect more data from the under-represented populations, re-weight what you have, or refuse to ship until the gap closes. Add human review at the boundary — for high-stakes decisions (loans, hiring, fraud flags), require a human in the loop, especially for the groups where the model is least accurate. Be transparent about the gap — publish the by-group numbers, including the bad ones, because hiding them is how the next team makes the same mistake. None of these are exciting. All of them work better than retraining once and hoping.

Check your understanding

Quick check

—

A fraud-detection model deployed by a Nepali bank flags transactions originating in Janakpur and Birgunj at roughly twice the rate of transactions originating in Kathmandu and Pokhara, even after controlling for transaction size. Which explanation is most consistent with what we know about bias in Nepal?

Madhesh customers are statistically more likely to commit fraud, and the model is correctly identifying this.
The training data was dominated by Kathmandu Valley transaction patterns, so the model treats Madhesh-region patterns as anomalous — producing a regional bias that is then reinforced by a feedback loop of more investigations.
The model is using a random number generator and the gap is coincidence.
Janakpur and Birgunj have slower internet, which the model interprets as suspicious.

Quick check

—

A hiring tool deployed by a Kathmandu firm is found to recommend male candidates from Khas-Arya surnames at noticeably higher rates than equally-qualified women or candidates from Dalit or Madhesi surnames. Which response is the most defensible mitigation?

Remove the “gender” and “caste” columns from the training data and continue using the model.
Audit the model’s acceptance and error rates split by gender, caste-cluster, and region; change the training data and add human review for under-served groups before re-deploying.
Rebrand the model as “AI-assisted screening” and ship it unchanged.
Add a disclaimer to the offer letter that the screening was automated.

What comes next

Having seen what bias looks like here, the obvious next question is the hardest one: what would it even mean for one of these models to be “fair”? It turns out that “fair” has competing mathematical definitions that cannot all hold at once — and choosing between them is a values judgment, not a calculation. The next section walks through what those definitions are and what choosing among them actually commits you to.