User Interviews: A Guide for Product Managers
Learn the risks and pitfalls of insufficient methodology in user interviews. Learn how cognitive biases like overconfidence can skew UX research outcomes, and discover the importance of rigorous practices in gathering accurate user insights.
User interviews have become a cornerstone for deriving insights into product development and UX design. However, their effectiveness hinges on rigorous adherence to established methodologies. Without this, the risk of succumbing to cognitive biases increases, potentially skewing the data collected and leading to poor decision-making.
This article is an excerpt from a full-length guidebook. If you like the ideas explained below, you can purchase the full PDF guide at the link below
Understanding the Dunning-Kruger Effect in User Interviews
The Dunning-Kruger effect is a cognitive bias where people with limited knowledge or competency in a domain overestimate their own abilities. In the context of user interviews, this can lead interviewers to believe they understand user needs thoroughly without proper training or adherence to research methodologies. This overconfidence can lead to significant errors and oversight in gathering and interpreting user data.
Cognitive Biases You Might Already Have And Not Know
- Confirmation Bias: This occurs when interviewers seek information that confirms their pre-existing beliefs. For example, if you assume users find an interface intuitive, you might ignore comments suggesting otherwise. This bias can be countered by asking open-ended questions that encourage diverse responses and by being receptive to unexpected answers.
- Recency Bias: Giving undue importance to the latest information received can overshadow earlier data. If a particularly compelling opinion is shared at the end of a series of interviews, you might prioritize it over earlier, equally valid feedback. Overcoming this requires a systematic review of all data, not just the most recent.
- Social Desirability Bias: Participants may alter their responses to align with what they think is expected of them. To reduce this, ensure anonymity where possible and create a non-judgmental interview environment that encourages honest feedback.
- Anchor Bias: This happens when too much emphasis is placed on the first piece of information offered, which then colors all subsequent data. To avoid this, interviewers should consciously maintain neutrality and consider each piece of data independently.
User Interviews - A Guide for Product Managers
This is a comprehensive guide designed to help Product Managers conduct effective user interviews, a critical component in the user research process. This document covers essential aspects of user interviews, including types of User Interviews, how to prepare for interviews, how to conduct them and then analyze them, and how to report the findings.
Do the new methodologies really work?
In recent years, many companies have enthusiastically adopted new "methodologies "aimed at integrating ongoing user feedback into product development. However, this approach has often been applied without sufficient adherence to a robust methodology. They are more theory than practice.
These approaches have been touted as a revolutionary framework by product teams unfamiliar with the deep-rooted practices of user experience research (UXR). Unfortunately, this has led to a superficial application that lacks the rigor and depth necessary for genuine insight.
While conceptually sound with its focus on customer-centric research and cross-functional collaboration, often fails in execution. They skim UXR best practices, resembling a reinvention of established methods without a proper foundation.
This results in a diluted process where:
- Multiple team members crowd the interview sessions, disrupting the rapport-building essential for genuine feedback. Proper collaboration should occur at the beginning when setting goals, and later during the analysis and reporting phases—not in the midst of sessions.
- The frameworks suggest that anyone, regardless of their training in UXR, can conduct meaningful user research, which undermines the specialized skills required in this field. This notion trivializes the years of learning needed to master techniques such as framing, mirroring, forming research questions, hosting interviews, adapting to various situations, probing, and coding data.
- Companies might cut costs thinking they are conducting research correctly, yet they end up biasing participants, asking leading questions, not properly analyzing data, or skipping essential workshops for problem discovery. The depth needed to truly define user problems and needs is sacrificed for speed, leading to ill-founded product decisions.
- Teams employing these methods often skip vital steps such as creating empathy maps, defining and writing problem statements, or properly investigating How Might We (HMW) questions. They rush into identifying "opportunities" without investing the necessary time to truly define the problems, which is a critical yet challenging step because of the rush to move into solution development.
Ultimately, for companies that are extremely tech and product-driven, any methodology that encourages the acknowledgment of user research could be seen as beneficial. However, with improper execution, these companies risk obtaining incorrect data, creating the wrong products, and ultimately failing.
The 'one-size-fits-all' approach neglects the nuances of professional UXR, where significant training and experience are crucial for effectively framing questions, managing interviews, and analyzing data.
Oversimplifying UXR to make it appear accessible without proper training these modern methodologies undervalue and potentially sideline professional user researchers.
A counterargument might claim that such frameworks offer a quick way to gather insights, preventing teams from getting stuck in perpetual research phases. If that's the case, it likely points to poor UXR practice or mismanagement rather than an inherent flaw in the research process itself.
Proper UXR is meant to inform and enhance product development, not to serve as a never-ending cycle without actionable outcomes.
The HOW of User Interviews
User interviews are indispensable in obtaining genuine user insights, which are crucial for informed decision-making in product development. However, conducting these interviews without a structured approach can lead to several pitfalls including misguided product decisions, biased responses, unprofessional demeanor in front of users, and potentially damaging user relationships.
Types of User Interviews
- Structured Interviews: Highly scripted, less favored for in-depth insights.
- Unstructured Interviews: Conversational and exploratory, ideal for experienced researchers.
- Semi-Structured Interviews: Balance of predetermined questions and flexibility, recommended for most needs.
This short guide is for Structured Interviews
Preparing for Interviews
- Define Objectives: Clearly articulate goals and tailor questions accordingly.
- Select Participants: Choose a diverse group to avoid biased data.
Conducting the Interview
! Build Rapport
Begin each interview on a gentle note to make the participants feel at ease. Establish rapport using both verbal cues (tone, pace, volume) and non-verbal cues (body language, facial expressions).
Analyzing the Data
Thematic Analysis
Identify and code recurrent themes across the interviews. This method helps in recognizing patterns that may indicate broader user behaviors or sentiments.
Reporting Findings
Focused Reporting
Only report findings that are directly relevant to the initial research questions. This ensures clarity and applicability of the insights gained.
Dovetail: Implement platforms like Dovetail to organize, analyze, and store insights efficiently. Such tools can help synthesize data from multiple interviews, aiding in the identification of patterns and themes.
They offer a free version, but the paid one is highly affordable - $29 User/month billed yearly or $39 billed monthly. Worth every penny!
Dovetail
Source
Maria Rosala
