Try : Insurtech, Application Development

AgriTech(1)

Augmented Reality(21)

Clean Tech(9)

Customer Journey(17)

Design(45)

Solar Industry(8)

User Experience(68)

Edtech(10)

Events(34)

HR Tech(3)

Interviews(10)

Life@mantra(11)

Logistics(5)

Manufacturing(3)

Strategy(18)

Testing(9)

Android(48)

Backend(32)

Dev Ops(11)

Enterprise Solution(33)

Technology Modernization(9)

Frontend(29)

iOS(43)

Javascript(15)

AI in Insurance(38)

Insurtech(66)

Product Innovation(58)

Solutions(22)

E-health(12)

HealthTech(24)

mHealth(5)

Telehealth Care(4)

Telemedicine(5)

Artificial Intelligence(152)

Bitcoin(8)

Blockchain(19)

Cognitive Computing(8)

Computer Vision(8)

Data Science(23)

FinTech(51)

Banking(7)

Intelligent Automation(27)

Machine Learning(48)

Natural Language Processing(14)

expand Menu Filters

Response Biases in User Research: A Guide for Culturally and Behaviorally Relevant Insights

Illustration by: vectorjuice  Source: Freepik

Introduction:

The success of businesses, design studios, and advertising agencies in India depends on effective user research. Response biases, however, can skew research results and prevent the creation of solutions that are culturally appropriate. This blog examines seven response biases that are common in user research, with examples from the startup, design, and advertising sectors. We will also go over methods for incorporating and avoiding these biases, ensuring inclusive research that is sensitive to cultural differences.

  1. Social Desirability Bias:
    Definition: When participants give comments they believe to be socially acceptable rather than revealing their real thoughts or behaviors, social desirability bias takes place.

    Example: Participants in market research for a sustainable fashion business may exaggerate their dedication to sustainable practices in order to conform to social expectations.
    And most of you may recall all those controversial advertisements during the holiday season. Tanish ad controversy, Surf Excel’s Holi ad controversy, Eros Now Dussehra Ad controversy. These are the outcomes of design decisions made by undermining social consent.
    At the same time, if this bias is understood and used properly, it can help the business in great ways.

    Techniques for incorporating and avoiding social desirability bias:
    Emphasize anonymity: Assure participants that their comments will be kept private to foster a comfortable environment where they can express their true feelings. Use terminology and phrasing that is culturally sensitive and resonates with Indian culture to enable participants to speak freely.
    Triangulation using behavioral data: To verify participant claims, combine survey replies with unbiased data from a real purchase or usage behavior.
  2. Confirmation Bias:
    Definition: Participants may exhibit confirmation bias when they choose to interpret data in a way that supports their pre-existing ideas or preconceptions, which may skew the results of the research.

    Example: In a user interview for a graphic design project, participants might only discuss the good features of their preferred design approach rather than considering other points of view.

    Techniques for incorporating and avoiding confirmation bias:
    Encourage participants to think about a variety of design methods and styles with well-balanced questions to encourage a more receptive exploration of ideas.
    Active listening: Maintain a nonjudgmental, impartial demeanor throughout interviews so that participants can share their opinions without feeling pressured to agree.
    Selecting a varied group of participants will help to ensure that various viewpoints are taken into account throughout the study process.
  3. The Hawthorne Effect:
    Definition: When participants are aware that they are being watched or examined, their behavior or answers change.

    Example: When focus groups are being held for an advertising campaign, members may make socially acceptable comments or alter their thoughts to reflect the group’s perceived preferences.

    Techniques for incorporating and avoiding the Hawthorne effect:
    Natural study environments: Gather data in situations where participants will interact with a product or service organically, resulting in more sincere and objective responses.
    Warm-up exercises or ice-breaking activities at the start of the session can help to create a calm environment and encourage people to express their true ideas.
    Multi-modal data collection: To gather unbiased insights, combine several research techniques like self-reporting, ethnographic observations, and remote monitoring.
  4. Anchoring Bias:
    Definition: Participants who heavily rely on the first pieces of information they encounter will have biased reactions and choices in the future.

    Example: Participants in price studies for mobile apps could base their perceptions of value on the costs of well-established rivals in the Indian market.

    Techniques for incorporating and avoiding anchoring bias:
    A number of references: Give participants a choice of pricing tiers and package options so they may assess the product’s worth on their own.
    Sensitivity to the perception of prices: When determining price ranges, take into account the participants’ cultural and socioeconomic backgrounds, as different market segments in India may have varied ideas of value.
    Comparative analysis: Ask participants to compare the proposed product or service with similar offerings in terms of features, benefits, and pricing to avoid solely relying on anchor points.
  5. Recall Bias:
    Definition: Recall bias happens when participants’ faulty or selective memories of the past cause their responses to be inaccurate.

    Example: Participants in user interviews for a meal delivery service can have trouble recalling specific instances of good or bad encounters, which could produce biased feedback.

    Techniques to counteract and prevent recollection bias:
    Stimuli and prompts: In order to ensure more precise and detailed feedback, use visual aids, screenshots, or prompts to help participants recall certain incidents. A timely study Conduct research right away after completing a task or encounter to record recent, vivid memories and lessen the need for retrospective memory. mixed-method strategy To verify and support users’ memories, combine self-reported experiences with behavioral information from app usage or transaction histories.
  6. Availability Bias:
    Definition:
    Availability bias is when participants’ responses are influenced by how quickly they can recall particular details or examples.

    Examples: Participants in mobile app usability testing could concentrate on well-known apps while ignoring lesser-known but equally valuable apps in the Indian market.

    Techniques for incorporating and avoiding availability bias:
    Contextual prompts: To promote a wider range of recollection and consideration,
    provide participants with specific scenarios or use cases pertinent to the Indian setting.
    Include participants with a variety of backgrounds, ages, and geographic locations to capture a wide range of experiences and preferences.
    Data triangulation: Combine self-reported experiences of users with information from app usage stats or market research to get a complete picture of user behavior and preference.
  7. Order bias:
    Definition:
    Order bias is the term used to describe the potential for participants’ replies to be biased depending on the order in which questions or tasks are given to them.

    Example: The sequence in which advertising themes are presented to participants may affect their preferences or evaluations.

    Techniques for incorporating and avoiding order bias
    Randomization: To ensure that any potential order effects are distributed equally across all participant groups, randomly order the presentation of ad concepts or design changes.
    Rotating designs: Apply a rotating design strategy in which various participants see the ideas in a varied order, enabling a balanced evaluation across the sample. Contextualization: Provide context and background information for each concept to ensure participants can evaluate each independently, regardless of the order in which they are presented.

Conclusion:
When conducting user research in India, it is important to carefully examine response biases as well as the relevant cultural nuances. Researchers can ensure more inclusive and culturally relevant findings by being aware of social desirability bias, confirmation bias, the Hawthorne effect, anchoring bias, recollection bias, availability prejudice, and order bias. Indian entrepreneurs, design firms, and advertising agencies can create products and campaigns that appeal to the diverse audience in India by putting into practice tactics like anonymity assurance, balanced questioning, natural research environments, cultural sensitivity, and data triangulation. Adopting these tactics would result in more useful and user-centric solutions, which will help businesses succeed in the competitive Indian market.

Further Reading: How To Prepare a User Interview Questionnaire

About the Author:

Vijendra is currently working as a Sr. UX Designer at Mantra Labs. He is passionate about UXR and Product Design.

Cancel

Knowledge thats worth delivered in your inbox

Machines That Make Up Facts? Stopping AI Hallucinations with Reliable Systems

There was a time when people truly believed that humans only used 10% of their brains, so much so that it fueled Hollywood Movies and self-help personas promising untapped genius. The truth? Neuroscientists have long debunked this myth, proving that nearly all parts of our brain are active, even when we’re at rest. Now, imagine AI doing the same, providing information that is untrue, except unlike us, it doesn’t have a moment of self-doubt. That’s the bizarre and sometimes dangerous world of AI hallucinations.

AI hallucinations aren’t just funny errors; they’re a real and growing issue in AI-generated misinformation. So why do they happen, and how do we build reliable AI systems that don’t confidently mislead us? Let’s dive in.

Why Do AI Hallucinations Happen?

AI hallucinations happen when models generate errors due to incomplete, biased, or conflicting data. Other reasons include:

  • Human oversight: AI mirrors human biases and errors in training data, leading to AI’s false information
  • Lack of reasoning: Unlike humans, AI doesn’t “think” critically—it generates predictions based on patterns.

But beyond these, what if AI is too creative for its own good?

‘Creativity Gone Rogue’: When AI’s Imagination Runs Wild

AI doesn’t dream, but sometimes it gets ‘too creative’—spinning plausible-sounding stories that are basically AI-generated fake data with zero factual basis. Take the case of Meta’s Galactica, an AI model designed to generate scientific papers. It confidently fabricated entire studies with fake references, leading Meta to shut it down in three days.

This raises the question: Should AI be designed to be ‘less creative’ when AI trustworthiness matters?

The Overconfidence Problem

Ever heard the phrase, “Be confident, but not overconfident”? AI definitely hasn’t.

AI hallucinations happen because AI lacks self-doubt. When it doesn’t know something, it doesn’t hesitate—it just generates the most statistically probable answer. In one bizarre case, ChatGPT falsely accused a law professor of sexual harassment and even cited fake legal documents as proof.

Take the now-infamous case of Google’s Bard, which confidently claimed that the James Webb Space Telescope took the first-ever image of an exoplanet, a factually incorrect statement that went viral before Google had to step in and correct it.

There are more such multiple instances where AI hallucinations have led to Human hallucinations. Here are a few instances we faced.

When we tried the prompt of “Padmavaat according to the description of Malik Muhammad Jayasi-the writer ”

When we tried the prompt of “monkey to man evolution”

Now, if this is making you question your AI’s ability to get things right, then you should probably start looking have a checklist to check if your AI is reliable.

Before diving into solutions. Question your AI. If it can do these, maybe these will solve a bit of issues:

  • Can AI recognize its own mistakes?
  • What would “self-awareness” look like in AI without consciousness?
  • Are there techniques to make AI second-guess itself?
  • Can AI “consult an expert” before answering?

That might be just a checklist, but here are the strategies that make AI more reliable:

Strategies for Building Reliable AI

1. Neurosymbolic AI

It is a hybrid approach combining symbolic reasoning (logical rules) with deep learning to improve factual accuracy. IBM is pioneering this approach to build trustworthy AI systems that reason more like humans. For example, RAAPID’s solutions utilize this approach to transform clinical data into compliant, profitable risk adjustment, improving contextual understanding and reducing misdiagnoses.

2. Human-in-the-Loop Verification

Instead of random checks, AI can be trained to request human validation in critical areas. Companies like OpenAI and Google DeepMind are implementing real-time feedback loops where AI flags uncertain responses for review. A notable AI hallucination prevention use case is in medical AI, where human radiologists verify AI-detected anomalies in scans, improving diagnostic accuracy.

3. Truth Scoring Mechanism

IBM’s FactSheets AI assigns credibility scores to AI-generated content, ensuring more fact-based responses. This approach is already being used in financial risk assessment models, where AI outputs are ranked by reliability before human analysts review them.

4. AI ‘Memory’ for Context Awareness

Retrieval-Augmented Generation (RAG) allows AI to access verified sources before responding. This method is already being used by platforms like Bing AI, which cites sources instead of generating standalone answers. In legal tech, RAG-based models ensure AI-generated contracts reference actual legal precedents, reducing AI accuracy problems.

5. Red Teaming & Adversarial Testing

Companies like OpenAI and Google regularly use “red teaming”—pitting AI against expert testers who try to break its logic and expose weaknesses. This helps fine-tune AI models before public release. A practical AI reliability example is cybersecurity AI, where red teams simulate hacking attempts to uncover vulnerabilities before systems go live 

The Future: AI That Knows When to Say, “I Don’t Know”

One of the most important steps toward reliable AI is training models to recognize uncertainty. Instead of making up answers, AI should be able to respond with “I’m unsure” or direct users to validated sources. Google DeepMind’s Socratic AI model is experimenting with ways to embed self-doubt into AI.

Conclusion:

AI hallucinations aren’t just quirky mistakes—they’re a major roadblock in creating trustworthy AI systems. By blending techniques like neurosymbolic AI, human-in-the-loop verification, and retrieval-augmented generation, we can push AI toward greater accuracy and reliability.

But here’s the big question: Should AI always strive to be 100% factual, or does some level of ‘creative hallucination’ have its place? After all, some of the best innovations come from thinking outside the box—even if that box is built from AI-generated data and machine learning algorithms.

At Mantra Labs, we specialize in data-driven AI solutions designed to minimize hallucinations and maximize trust. Whether you’re developing AI-powered products or enhancing decision-making with machine learning, our expertise ensures your models provide accurate information, making life easier for humans

Cancel

Knowledge thats worth delivered in your inbox

Loading More Posts ...
Go Top
ml floating chatbot