Researchers Ask Questions Consumers Can’t Answer

By George Anderson

Ron Sellers has spent 23 years conducting brand and advertising research and
in that time he has seen clients trying to get answers to questions that consumers
simply can’t answer properly.

In a recent article on the Brandweek website,
Mr. Sellers of Grey Matter Research & Consulting wrote, "In at least
a third of the advertising-related focus groups I’ve moderated, the client
has insisted we ask a question such as: ‘What would be the best media to advertise
our product?’ The inevitable answer is television. But that’s just because
TV is what people tend to envision when they think of advertising. It’s
not because consumers actually have deep insight that a TV buy would suit the
marketplace goals of the brand in question."

Questions about the effect advertising has on consumer purchasing decisions
can also lead to wrong answers. "Consumers will tell us flat-out that advertising
has no effect on what they buy. If only they realized how much it really does," he
wrote.

"If consumers don’t even know all the reasons for their own behavior," he
asked, "why do we continue to treat them as if they do, ask them questions
they cannot hope to answer accurately and rely on the results for critical
decisions?"

Discussion Questions: Is there any point to asking consumers
to consider the reasons behind their own shopping behavior? What is the most
effective means to understand the connection between advertising and consumer
shopping behavior?

BrainTrust

Discussion Questions

Poll

12 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Bob Phibbs
Bob Phibbs
13 years ago

Most of these shopping surveys, I feel, are flawed from the start because they overlay logical judgment on what essentially is a past emotion. Ask a guy why he married the girl, “When I saw her I just knew.” Is that logic? No, he had emotions arise, processed and moved on. Looking back all he can give is generalities. A little girl being scolded by her mother, “Why did you do that?” gets a shrug and an “I don’t know.” Why do we feel in marketing that pressing the point yields anything but vague memories that grasp at straws rather than providing insight?

Dr. Stephen Needel
Dr. Stephen Needel
13 years ago

There is lots of evidence to suggest that consumers do not think much about their purchasing behavior and when we force them to do so, we get answers like Mr. Sellers finds, not the truth. This applies to retail settings too–asking shoppers whether a shelf is easy to shop or poorly organized always gets you positive answers–they are easy to shop and nicely organized (no matter how badly they are organized).

We prefer behavioral research instead of asking people questions, whether it’s virtual reality for retail issues or split cable for advertising, we want to see how shoppers react behaviorally to our marketing variations.

Paula Rosenblum
Paula Rosenblum
13 years ago

It’s all in how you ask the questions.

Certainly a large part of purchasing behavior is emotional…but it is useful to understand what those emotions are. And these days, logic also plays a large part in purchasing decisions. It’s useful to understand the thought process as well.

But this is the glory of social networks. You can find out a wealth of information without creating focus groups. Customers are MORE THAN happy to tell you what they’re thinking and feeling. The challenge for retailers is to aggregate and distill that sentiment into something more quantifiable. It’s doable.

David Morse
David Morse
13 years ago

As a moderator myself, I’m a big believer that it never hurts to ask. But as researchers, we need to go beyond taking answers at face value. We need to interpret and read between the lines. And more importantly, we need to ask the right questions, and ask them in the right way.

It’s always a challenge. Clients tend to hear what it is that they want to hear. Also, they can’t be expected to have the expertise to read between the lines compared to those of us that do this for a living.

I attended some focus groups last night. We were testing a great idea for an entertainment company. Naturally, the respondents liked the idea–it’s a smart one. But they didn’t jump up and down either. Sometimes, you learn more from the silences and the way things are said. Good idea, but needs to be executed the right way. We learned a lot.

There’s a lot wrong with focus groups. “Group think” is a big issue. Respondents tend to say what they think you want them to say. After all, they are being paid. Ego, as it always does, plays a big factor.

But focus groups do have their place. They offer a great medium for generating discussion. A good moderator can challenge and go deeper. And a good moderator knows how to manage client expectations and reactions.

Focus groups are a tool. Like any tool, they have their place, but they also have limitations. You can’t fix a car with a screwdriver alone. But I wouldn’t want to be without one in my toolbox.

Joan Treistman
Joan Treistman
13 years ago

This article is like setting up target practice. If you don’t feel your questions get appropriate and valid answers, find different questions. And by all means consider a different research methodology.

I would never ask consumers what media should be used. However, they can tell me how they found out about a TV they’d like to buy, a car model whose showroom they visited, etc. They can tell me what they read, what shows they watch…and whether or not they DVR through the commercials.

If you’re talking with your target audience and learn about their media habits there’s an opportunity to use the knowledge for media strategy. And this is a rather cumbersome approach within the context of the focus groups Ron Sellers mentions. There are many more sophisticated techniques out there using consumer behavior, not opinions of the future and memories of the past.

It makes me very uncomfortable to think that readers of the article by Mr. Anderson believe that research = focus groups. As a moderator and research methodologist I know there’s a time and place for focus groups. And when they are good, they can be very very good…but it takes knowing the questions to ask and orchestrating the flow so that consumers can dig down into their experiences and feelings. But it’s not one-size-fits-all. There are times that transactional information is more relevant than a focus group, when online quantitative surveys offer better insight and reliability, when one on ones can provide creative insights unobtainable any other way.

So puhleeze, can we stop equating research with focus groups?

Ben Sprecher
Ben Sprecher
13 years ago

Mr. Sellers is exactly right. People can be terrible at describing the triggers for their own behavior or predicting their own future behavior. Just think how many New Year’s resolutions are made to absolutely, positively, with no excuses this time around, lose 20 lbs. on the new diet. By February 1, many of those great intentions have been thrown into the trash can along with the the crumpled ruins of jumbo bags of chips and cookies.

Other panelists here make some great points–how you ask the questions and how you interpret the answers is hugely important–but the fact remains, people are unreliable when it comes to self-evaluation.

Here’s what I want to see: I want to see focus group and survey results tied directly to behavioral data. I want researchers to run focus groups with shoppers whose loyalty card numbers can then be traced back to their answers. I want some of the questions asked in the focus group and on surveys to be specifically designed to identify bias between answers and behavior. Ask the consumer how often they buy salty snacks, and then VERIFY the answer against their actual shopping history. Ask what product features (organic, low fat, etc.) the consumers *think* are important to them, and then verify how much their shopping basket *proves* they are important to them.

With the reality check in hand, researchers will have a great tool with which to interpret the answers that consumers give in focus groups and on surveys.

Pamela Danziger
Pamela Danziger
13 years ago

I gain tremendous insight in the why behind consumer purchases simply by asking consumers about their shopping behavior. The more you engage people in dialogue, asking questions, asking for explanations, asking them to give you examples, the more can be inferred about their basic motivations and drives. Surely you can’t ask a consumer ‘why did you do this, buy that…whatever” and expect a staight answer. But by being genuinely interested people will tell you an awful lot about the why in their behavior.

Ed Rosenbaum
Ed Rosenbaum
13 years ago

I agree with Ron Sellers article. My responses falls in line with Bob Phibbs so there is no need to repeat.

I will add that today’s economic temperature has begun to have more input in consumer buying decisions. In the past emotion dictated buying decisions and logic justified them. Today, emotion still starts the thought process; but logic then steps in and says “Yes, we can buy this; but do we really need it?” It has become want vs. need.

Herb Sorensen, Ph.D.
Herb Sorensen, Ph.D.
13 years ago

For many years we have built our research on observation. The rule should be, observe first (so you know the “facts”) and then ask questions to possibly get more insight.

At the same time, I make a distinction between “hard” answers, like, “how many children do you have?”–where the answer is simple, unambiguous, and unlikely to be distorted–and “soft” questions like “is it too sweet, just right, or not sweet enough?” For this second question there are cross-currents due to a lifetime of societal input that “sugar is bad” conflicting with the general taste assessment that “sweet is good.” It is a fact that if you ask the more neutral type of question, like, “which of these three do you like best?”–you will find that the one that is liked best is also independently rated as “too sweet.”

This points up the problem of what I call “pretend market research,” a great deal of which ignores the context of the interview, as well as the filtering process of the respondent (bias toward providing a “rational” answer) and the overwhelming injection of the researcher’s own paradigm into the results.

This last point parallels the problem in historical research, where it has been noted that often historians write as if they were describing history as they might see it by looking at a reflecting pool at the bottom of a deep well. The only problem is, that a good deal of what they see is compounded with their own reflection. That is, what they see is at least as great a reflection of who they themselves are personally, as it is any part of objective historical reality. This is a massive problem in market research, little noted.

Dennis Serbu
Dennis Serbu
13 years ago

After reading Dr. Sorensen’s book “Inside the Mind of the Shopper,” I am further convinced that there is a great divide between what a shopper says and what a shopper actually does. Focus groups serve a purpose to further refine what really needs to be studied, but the best answers come from “in vivo.” Or am I thinking “in vino veritas”?

In the context of a shopper experience, a customer behaves in a different manner than in a theoretical environment such as a lab or meeting room. Is the test subject trying to please? As the great Yogi Berra is quoted, “sometimes you can observe things by just watching.” This needs to be done in the wild and preferably without knowledge they are being studied. The answers to all of our questions are out there. It is a matter of creativity on how to capture the information.

Mark Johnson
Mark Johnson
13 years ago

You should use behavioral data first as the baseline and then use the overlays to add more depth.

Craig Sundstrom
Craig Sundstrom
13 years ago

I think anyone who has ever answered a survey where they were asked “which of these 23 brands of shampoo makes you proud to be an American?” (or whatever) can sympathize with this problem, but ultimately what is–or isn’t–learned from a survey depends on the skills of the surveyor…even a long string of “I don’t knows” tells something. My bigger concern is that as (presumably) the internet makes survey taking more common, the process begins to affect testee behavior: “Oh I think I’ll try Brand X, it’ll look good on the next survey I take…” is something I fear is happening.