How Leading Questions Lead to Biased Market Research
by Mike Poirier
Market researchers need to be critical of survey design, because we’re knee-deep in it every day. DIY survey solutions have emerged as a cost-effective solution for companies to obtain consumer feedback. However these solutions lack the expertise of market research professionals. They tend to allow for unnecessary errors in survey design. One of the most dangerous and prevalent issues that lead to bias errors is asking leading questions. I’m going to show how leading questions can adversely affect responses so that they generate biased market research. Then I’ll show demonstrate how to rephrase questions so that they no longer promote bias.
A leading question is one that influences an individual to provide a particular response. The point of survey research is to obtain objective consumer feedback. It is crucial that questions are clearly crafted so answers are based solely on respondents’ perspectives.
Here are a couple examples of leading questions I’ve seen in surveys over the past few years.
“Are you more likely to purchase Brand ABC over others because of your experience?”
This question was in an on-site intercept survey at an event marketing program. “Are you more likely” is leading. It’s worded like a “yes/no” question, but it influences respondents to think in terms of being “more likely” to purchase. Responses are biased toward the brand. The brand team may look like rock stars when the program is finished, but they’ll never know how much of the reported purchase intent was elicited from the wording of the question rather than the respondent’s true perspective.
Here is a more objective way to word that question:
“How likely are you to purchase Brand ABC over other brands because of your experience?”
The respondent is not guided to give a positive or negative response. The answer is elicited from a purely personal attitude. Responses can be analyzed using a Likert scale. We usually use a five-point scale, with five being “Extremely Likely” and one being “Extremely Unlikely.” There’s merit to using a seven or a ten-point scale as well, but five fits well with brief, on-site intercepts. See an earlier post of mine on eliminating interviewer bias.
More Examples from Qualitative Research
Leading questions are also a problem in qualitative research. Here is an example from Herman & Bentley, 1993:
“How good was the treatment your baby got at the health center?”
“How good” is leading. It suggests that “good” treatment part of the expected answer. Respondents may avoid providing constructive feedback based on the wording of the question.
A more objective way to ask the question is as follows:
“How do you feel about the treatment your baby received at the health center?”
Like the previous example, the respondent is encouraged to answer objectively, based solely on his or her experience.
If you’re looking for help obtaining objective consumer feedback, reach out to us. If you’re going to use a DIY solution, heed best practices by designing questions that eliminate the possibility of biased market research.
Photo Source: https://www.flickr.com/photos/luckyno3/