Why are retailers turning to passive data for consumer insights?

Through a special arrangement, presented here for discussion is a summary of a current article from MarketingCharts, a Watershed Publishing publication providing up-to-the-minute data and research to marketers.

Custom surveys are considered the most important data source for the creation of insights today, but fast forward two years and they will be rivaled by passively collected consumer data, according to GfK’s "Future of Insights" report.

Indeed, 28 percent of market researchers surveyed feel that passively collected consumer data (i.e., tracking location, website/online behavior) will be the most important data source in two years’ time, more than double the proportion (13 percent) who feel that it is currently the most important. That will put it virtually on par with customer surveys. Twenty-nine percent expect consumer surveys to become the most important data source in two years, down from 43 percent currently.

Meanwhile, social media and digital data (search engine, website visitation data) are also slated to become more important over time, while point of sales/sales data will decline in importance.

Insight creation

Source: MarketingCharts

Other findings from the study:

  • When asked what is the "biggest gap" in the market research industry today, suppliers were more likely to cite "data quality" (20 percent vs. 15 percent for clients). While "integrating information from different sources to tell a story" was the most frequent choice among both groups, the percentage for clients (31 percent) was significantly higher than for suppliers (23 percent).
  • Budget limitations are and will likely remain the leading organizational issue for clients and suppliers, but a variety of other concerns — from data integration to regulatory concerns — are seen as nearly equal in importance.

Among the 709 respondents, 315 were clients and 394 were suppliers.

BrainTrust

"Retailers are beginning to understand that knowing how shoppers actually shop through tracking data and other observational means has much more utility than asking shoppers in a survey about their intentions and attitudes."

Mark Heckman

Principal, Mark Heckman Consulting


"I certainly see passively-collected data rivaling customer surveys in the future. A great benefit to this is that it is passive — all customers participate and that provides a much more realistic sample than those who respond to customer surveys."

Zel Bianco

President, founder and CEO Interactive Edge


"The problem is that the overwhelming majority of shopping decisions are made by the subconscious, and when we survey shoppers about those behaviors, they can’t be aware of the subconscious factors that influenced the behavior."

Jason Goldberg

Chief Commerce Strategy Officer, Publicis


Discussion Questions

Do you see passively collected data rivaling consumer surveys within two years? What do you see as the upside and downside of passively collected consumer data versus consumer surveys?

Poll

18 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Ralph Jacobson
Ralph Jacobson
8 years ago

I see what is described as “passive” data in this article already becoming more valuable as we speak. This is happening now. Structured and unstructured data from myriad sources, including social, news, events (sports, etc.), weather and others are now being captured by innovative retailers around the world. The challenge is to be able to synthesize any meaningful and actionable insights from this information. The good news is that there are some emerging technologies available today that are doing just that. And some great retailers are helping develop these apps!

Dr. Stephen Needel
Dr. Stephen Needel
8 years ago

It’s not either/or, it’s both and more, as one of my professors used to say. First, ignore the predictions — they are never right. Second, they often answer very different questions. If retailers are changing the questions and the questions can be answered passively, then yes, it will grow. If retailers want to know what shoppers think about their stores, passive data is unlikely to give them much.

Mark Heckman
Mark Heckman
8 years ago

Retailers are beginning to understand that knowing how shoppers actually shop through tracking data and other observational means has much more utility than asking shoppers in a survey about their intentions and attitudes.

Given shoppers’ inherent inability to predict their own behavior, I am not surprised to see that retailers and brands are leaning towards learning about actual behavior, as it happens in the store, versus passive information. That is not to say that shopper surveys are without their place, as they still can serve as a great tool to track long term trends and other attitudinal issues.

In my view, retailers that embrace capturing “active” shopping behavior are going to have a clear competitive advantage in merchandising, store layout and product placement and — most importantly — satisfying the shoppers.

Zel Bianco
Zel Bianco
8 years ago

I certainly see passively-collected data rivaling customer surveys in the future. A great benefit to this is that it is passive — all customers participate and that provides a much more realistic sample than those who respond to customer surveys. However the insights and action steps that customer surveys provide will always be significantly more straight forward than passive data and figuring out what to do with that data once you have collected it is sure to be an interesting problem.

Jason Goldberg
Jason Goldberg
8 years ago

In shopper marketing, surveys have never been a very reliable or valid research methodology. The problem is that the overwhelming majority of shopping decisions are made by the subconscious, and when we survey shoppers about those behaviors, they can’t be aware of the subconscious factors that influenced the behavior.

“Observed preferences” are far more valuable than “stated preferences” and those observed preferences are increasingly easily collected via passive or implicit means.

J. Peter Deeb
J. Peter Deeb
8 years ago

Match the various forms of data that are not surveys and you see that actual behavior will be much more closely tracked and give a much more robust set of consumer insights whereas surveys and interviews, etc., must be vetted for “human” intervention and desire to please.

This information will be much more accurate and can lead to more effective and efficient marketing by both retailers and manufacturers alike. Several things are necessary for this to happen including investment in technology and analysis, as well as marketers who can convert these insights into good marketing programs.

Joan Treistman
Joan Treistman
8 years ago

The primary reason passive data will overtake consumer surveys is because marketers don’t understand the limitations of passive data given the decisions they have to make. It’s cheaper, faster and more defensible. It can also be irrelevant, inappropriate and unreliable. But if you don’t know what you don’t know, you make decisions based on what you have in your hand.

I agree with Stephen that both consumer surveys and passive data are needed, perhaps together or simply at different times for different purposes.

I’m currently working with a client to reach out to their Facebook fans to answer some questions to make informed decisions regarding changes for the brand. But the respondents to my survey have to be defined. I have to find out who those fans are. Are they current users of the brand, past users, frequent or infrequent users, prospects, advocates, where do they live, etc.? I need to know all that in order to properly frame any survey results and recommendations. I won’t just throw out questions and aggregate answers by whoever.

We’ve lived with transaction/sales information for decades, some of it internal data and some externally sourced. But that’s always been how we look in the rear view mirror and predict future behavior based on the past. We don’t have to limit ourselves to that.

But if marketers don’t know what they don’t know, they’ll go for cheaper, faster and more defensible. It just may not help them make good decisions.

Brian Numainville
Brian Numainville
8 years ago

This isn’t as simple as picking one method or another. Integrating surveys, social media, passive data, online behavior and more can yield great insights. But each method has its strengths and weaknesses and there is a place for each, now and into the future.

Shep Hyken
Shep Hyken
8 years ago

We always need to know how our customers feel, so there will always be customer surveys. And, there is other data that will help us with trends that will help us know what will sell, how much inventory to hold, etc. Both are important. They give us different angles, and ideally will align with each other.

Ian Addie
Ian Addie
8 years ago

As a provider of passive data (in-store behavioural observation) we often work in partnership with agencies providing survey-based insights. Our observations provide the WHAT to the surveys’ WHY, but beyond that controlled tests coupled with observation can give unique insight into what is going on in-store, what works and what doesn’t.

Whereas surveys are often based on the opinions of a few hundred shoppers at most, we are not constrained in the same way and can analyse the behaviours of thousands meaning even small shifts are statistically significant. Retailing is a volume business where even a 1 percent uplift in purchasing can mean a lot to the bottom line, but surveys with their low base sizes, even if the answers that shoppers provide are reliable, are unable to provide the confidence that shifts of this magnitude are real and not sampling anomalies.

Doug Garnett
Doug Garnett
8 years ago

Data collection (passive) is on the typical curve of a fad … The big operations have been shifting for years toward the mythology that they’ll learn far more because this data is “real.” What we forget is that data is rife with errors. But worst of all, observed data has severe limitations. And it is extraordinarily dangerous for management because we NEVER know what we don’t know.

My favorite example is what I learned about dinosaurs growing up. Dinosaurs were all reptiles, all loners, all with strange scaly skin, etc. But now my kids learn that many dinosaurs evolved into birds, that some lived in groups, there’s evidence some raised their young, some even had feathers at the time or the beginnings of feathers.

What changed? The best paleontologists drew their theories from the observed data. But they never knew what was missing from that data. Over time, new bits of data emerged and we found out their conclusions were quite far off.

There’s tremendous risk to the retail executive team who trusts observed data too much. Like these paleontologists, they’ll never know what they DON’T know from the data. And that leads to massive error.

Direct survey and exploratory work with customers should remain the basis of executive decision making, and found data from data collection should be added as one more source. But right now, in the midst of the hype of a fad, executives are being told to throw out all that other stuff and simply trust this highly-faulty data.

For a good book showing how severe the consequences can be, I recommend “The Fires” by Joe Flood, documenting the NY Fire Department’s abuse of decisions by data collection in the 1970s that led to horrific loss of life and destruction of entire neighborhoods.

Steve Montgomery
Steve Montgomery
8 years ago

The importance of the data will first and foremost depend on the two factors — the intent of the information and its quality. Passive data might tell what someone did but not why. Customer surveys are often aimed at collecting information on what customers did (or what they say they did) and why (or what they say was the reason). The combining of the two is likely to yield better results than does either separately.

Peter J. Charness
Peter J. Charness
8 years ago

I keep waiting for a “leave me alone” trend to hit consumers. Through the active collection of data/surveys and pointless emails, retargeting advertising verges on irritating to outright disturbing. Someday there may be an “enough is enough” moment in which case the passive data becomes the “actuals” in any case.

Gordon Arnold
Gordon Arnold
8 years ago

The battle for supremacy between passively collected consumer data and survey data is in desperate need of a place to fight. For the past 15 years, consumer economic and geographical demographics have been in severe turmoil. At the present time, the changes have spread world wide and most likely will continue for a few more decades.

There is some direction in all this evolution and some points of observation that can keep retailers informed. Point-of-sale is where we will find the most reliable information about our customers. From point-of-sale we know a lot about what our customers do to purchase from us.

Becoming fixated on the information we can not measure effectively has become paralyzing in our efforts to effectively address market needs and wants. Looking for opportunities to enhance the customer experience with better product and service in front of the corporate response is how we need to posture ourselves. This, smothered in ease of participation and security, will lure in more consumers if only to give it a try. It is a time to focus on what we can do with the information we can count on.

Lee Kent
Lee Kent
8 years ago

So much of shopping is emotional so, how does one capture the emotion that compelled a purchase from a survey?

Just think about it. If a retailer runs different messaging at different times or for different customers and can see an uptick in purchasing related to this, that is one step closer to capturing that emotional purchase.

And what if ones friends are buying a certain brand? Don’t retailers want to know that?

Passive data is a must in this day and age. Actions speak louder than words!

And that’s my 2 cents.

Arie Shpanya
Arie Shpanya
8 years ago

It’s the difference between what consumers say they have done/will do and what they actually do. The latter is much more useful now and it will continue to be in two years.

Kenneth Leung
Kenneth Leung
8 years ago

With internet of things and mobile technology, “passive” data collection is becoming better at observing actual behavior than consumer surveys. Problem with survey data is that human recall is inheritly inaccurate when it comes to time and space, while it is more accurate in the emotional data. I think retailers will rely more on passive data for spatial and chronological data, and rely on survey for emotional and also cross-check decision making data. Asking how many times the customer shopped and cross checking against the passive data of how many times they did shop can yield interesting insights.

Patricia Vekich Waldron
Patricia Vekich Waldron
8 years ago

Absolutely agree that “passive data” is a valuable — and an increasingly available — source of insight. Retailers are foolish to not take advantage of location, weather, social and other information that can be combined with sales, marketing and customer data. The Fung Group’s Explorium is a great example of how to turn insight into action.