Chapter 05 · Section II · 14 min read
Bias and cultural blindspots
Where the model produces "technically correct" output that quietly gets Nepal wrong — and how to spot it before it goes out.
Hallucination is the loud failure mode of generative AI — visibly wrong things you can point at. Bias and cultural blindspots are the quieter one — output that is technically correct, fluent, and confident, but that gets your specific context subtly wrong in ways the model cannot see.
For a Nepali user, this is the failure mode that matters most. The training data behind every major model is overwhelmingly Western, English-language, urban, and middle-class. When you ask the model for help with a task in your context, it draws on a distribution that may not include yours.
A simple example
You ask a model to generate “an image of a typical family enjoying a holiday meal.” Without instruction, you almost certainly get: a white nuclear family of four around a turkey-and-cranberry meal in a suburban Western kitchen.
This is not the model being biased against Nepali families. It is the model producing the most statistically common image given the training data. The default is the Western default because the training distribution is Western. The Nepali family was always available; it just wasn’t the most likely output.
The fix is small: “a Nepali Brahmin family of seven enjoying Dashain tihar khana at a traditional Newari home in Bhaktapur, Aamaa serving sel roti, the whole house lit by diyo.” Specify, and the model can produce it. Don’t specify, and it falls back to the training distribution’s centre.
Where bias shows up in text
Subtle text patterns to watch for:
Names. Ask for “five sample names for a story character” and you’ll usually get five Western names — “Sarah, James, Maria, David, Lisa.” The fix is again to specify: “five Nepali names — three women, two men, from different ethnic communities.”
Examples and analogies. When explaining a concept, models reach for examples that resonate with their training distribution — American baseball, European history, Silicon Valley startups. For a Nepali audience, you can ask: “Explain this concept using Nepali examples — Khalti, Pathao, or paddy farming.”
Tone and formality. Models trained on Western English have an American business-casual default. Nepali correspondence often calls for more formal honorifics, more relational context, more deference to seniority. Specify or the output will feel oddly flat.
Implicit assumptions about households. Western-default outputs assume nuclear families, individual decision-making, urban infrastructure, separate residences for elders. None of these is universally true in Nepal. Outputs about home life, family decisions, and elder care will quietly mismatch unless you specify.
Money and salary references. Models default to dollars or euros. For Nepal-specific content, specify Rs. and a realistic local salary range.
Geographic defaults. “A bustling city” defaults to New York or Tokyo, not Kathmandu’s Asan. “A rural area” defaults to American Midwest, not Karnali.
None of these are bugs. They are probability defaults. Knowing they exist is most of the battle; the remaining battle is consistently specifying.
Where bias shows up in images
Image bias is more visible, and often more revealing.
“A doctor.” Western man, white coat, stethoscope. Default. For Nepali context, specify gender, ethnicity, and setting.
“A professional.” Western business suit, modern office. Specify if you want kurta-pyjama in a small office in New Road, or a teacher in a daura suruwal at a school in Solukhumbu.
“A wedding.” Default Christian wedding — white dress, dark suit, church. For a Nepali wedding, specify the community (Hindu, Buddhist, Newari, etc.), the location (mandap, monastery, courtyard), and the attire.
“A traditional Nepali home.” Output: an aesthetically Nepal-coded image — terraced fields, mountains, prayer flags — but architectural specifics often wrong. A traditional Brahmin home in Gulmi looks very different from a traditional Tharu home in Bardiya; the model will pick whichever stereotype is most common in training data, usually the mountain-tourism image.
“A child.” Western default. For a Nepali child, specify ethnicity (Khas, Madhesi, Tharu, Newari, etc.) and clothing.
The model is generating aesthetically plausible outputs — they look like Nepal in the same way a tourism poster looks like Nepal. Local specificity requires effort.
Where bias shows up in voice and video
The same patterns hold but with extra failure modes.
Voice synthesis. Nepali voices have improved but most defaults are urban Kathmandu Nepali. Regional accents, the rhythms of Mithila Nepali, the prosody of Magar-influenced speech — these are either missing from the synthesis options or sound forced.
Video. Default people, default settings, default body language. A “Nepali wedding video” generated without specifics will look like a stock Indian or Bangladeshi wedding video, not a Nepali one. The visual vocabulary is borrowed from neighbouring countries, not learned from Nepal specifically.
Bias in extracted decisions
The most consequential category. When a generative model is used to assist with a decision — screening job applicants, assessing creditworthiness, evaluating an essay, deciding which customer to prioritise — biases in the model’s pattern recognition translate into biased decisions.
Concrete examples:
- A model used to summarise resumes will rate Western-sounding names and English-medium education more favourably than Nepali rural backgrounds.
- A model used to “grade” student essays will rate Western rhetorical structures more highly than Nepali ones.
- A model used to evaluate customer support ticket urgency may under-rate concerns expressed in less assertive tones common in some Nepali communities.
These are subtle, hard to detect from a single output, and they compound. If you are using generative AI in any decision-affecting context, you need to:
- Be aware that bias is present and may affect outcomes.
- Audit by feeding in known-equivalent inputs and checking that outputs are equivalent.
- Disclose the use of AI to people affected by the decisions.
- Provide a path for human review.
We will return to this in Chapter 6, but the principle is worth stating now: the higher the stakes of a decision, the less appropriate it is to delegate the underlying judgement to a generative model.
A working habit: specify aggressively
A pattern that pays back many times. Before any task with cultural specificity, take 30 seconds to specify the context explicitly in your prompt.
A small library to draw from:
- “The audience is Nepali, primarily Kathmandu-based, generally bilingual Nepali-English.”
- “Use Nepali examples — Khalti, Pathao, daal-bhaat, OCR, Devanagari.”
- “Use names from a range of Nepali ethnic communities.”
- “Refer to prices in NPR.”
- “Tone: formal Nepali correspondence, appropriately honorific.”
- “Setting: assume a typical Kathmandu small business, not a Silicon Valley startup.”
These additions cost almost nothing in prompt length. The output difference is immediate and noticeable.
Check your understanding
Quick check
—You ask an image model for “a typical wedding” without further specification. The output is most likely to be:
What comes next
We’ve covered the two main failure modes — hallucination and bias. The next section is about the situations where you should not reach for generative AI at all, no matter how careful your prompting and verification.