Why has retail’s transition to data-driven enterprises been so arduous?

Jul 26, 2016
Michael Day

The ability to understand, predict and ideally shape consumer behavior lies at the heart of today’s heightened interest in analytics and the growing appreciation for the huge potential of data-driven insights. The retail data revolution started when Walmart launched Retail Link and data-driven supply chain management more than 20 years ago. This has been carried forward with a vengeance by Amazon, which is leveraging data to understand what prices to list, customer paths to purchase and monetizing insights.

In the recently released “2016 Retail and Consumer Goods Analytics Study” from RIS News, retailers consistently ranked analytics as a strategic priority. Retailers now know that data management is the core foundation of getting things right. They know that uncovering and acting on data-driven consumer insights is essential to stand out in a crowded market place battling for a well informed, highly connected and technology empowered consumer.

However, efforts towards improving analytics maturity are to-date unimpressive and underwhelming. In the RIS News study, retailers are equally split regarding the two primary challenges they claim are keeping them from adapting to analytics and becoming more data-driven:

  1. Difficulty shifting away from a culture that has relied on intuition rather than data.
  2. The absence of clearly a articulated analytics strategy in most retail organizations.

The time for retailers to act is now. While access to data scientists is tight, the basic enterprise analytics solutions available are relatively inexpensive compared to the potential return on the investment. Perhaps more importantly, the lack of investment in this critical technology may result in a competitive disadvantage that negatively impacts the company for years.

DISCUSSION QUESTIONS: Do you agree that retailers have been too slow to leverage analytics compared to other industries? Has being data-driven and analytics-focused become operationally essential for retailers looking to compete with Amazon.com and other rivals?

"Using analytics to drive key business decisions may very well define the winners and losers of retail. The bar has been raised."
"There simply are too many choices and the pitches by these companies are too complicated to understand their differences and their value."
"We are seeing more retailers adopting the data analysis tools in centralized, single versions with access across all key departments and regions."

Join the Discussion!

21 Comments on "Why has retail’s transition to data-driven enterprises been so arduous?"

Notify of

Sort by:   newest | oldest | most voted
Peter Sobotta

There is no question that retailers have been slow to leverage analytics, but with unproven ROIs it is a big step to take. Early adopters are companies that have a data-driven culture and believe. Hence why we are seeing e-commerce companies among the first to leverage their data, and with great success. They have the advantage of being born into an environment where the customer experience drives data-driven decisions. Old school retailers will come around, but it may be too late for many of them.

Using analytics to drive key business decisions may very well define the winners and losers of retail. The bar has been raised.

Sterling Hawkins

There are two sides to the data question: product data and consumer data. Retailers were driven to leverage product and logistical data to keep up with Walmart in the late 90s and have done a pretty good job staying on top of the technology on that side of the house. It’s the consumer side that has been lacking. The ironic part is that most every retailer out there has a mantra of “focus on the customer” while they’re not in a position to use data to back it up. I’m with Peter that consumer data — and the enterprise-wide application of it — will define the winners and the losers. Especially as the online and offline worlds continue to merge.

Kim Garretson

I believe the perceived slow leveraging of analytics has emerged from one factor: that there simply are too many choices and the pitches by these companies are too complicated to understand their differences and their value once implemented. Consider this recent headline from LinkedIn: “400+ VC’s poured over $11 Billion into 323 marketing tech startups in 2015”.

Also, this excerpt from Venture Beat. “It turns out that marketers are spending well over a third of their budgets (on average) on analytics. This in spite of the report finding that levels of confidence in analytics’ ability to generate insight are mediocre, at best.”

Ian Percy

You know, sometimes the problem is the answer.

This will be anathema to some readers, but perhaps retail shouldn’t abandon their intuition and normal human communication for technology and data. Maybe the slowness to do so is the universe telling retailers that it’s not the way to go.

As I was reading this I had the picture of customers and retailers in a data-duel. Customers are using their data tools to define and find their preferred retailer. Retailers are using their data tools to define and appeal to their preferred customers. That’s like a dating service where two people stare at each others’ picture and review data but never actually meet, or have dinner, or a dance or a hug. No one is going to say “Let’s be a couple, I love your data.”

Relationships ARE intuitive — of the heart. I’d say we need more of that not less.

Cathy Hotka

I have near-weekly meetings with retailers and have been asking this question for over a year. Their answer is remarkable: while nearly all have multiple analytics packages, they are silo’ed by department, without a master plan for execution or vision for what could be accomplished. It’s hard to accomplish something without a plan.

Ross Ely

Most retailers understand that analyzing their shopper data and applying the insights to their business can have a profound impact on their growth and profitability. However, they also perceive that these solutions are complex and expensive. Retail is a low-margin business and retailers are consequently risk-averse when evaluating new technology. The reality is that analytics solutions today are affordable and easy-to-use, and can be highly beneficial to retailers in enhancing their competitiveness and value to shoppers.

Phil Rubin
1 year 4 months ago

If you need evidence that retailers have been too slow to be leverage analytics and be more data-driven, just look at their stock prices. There is an increasingly clear and sizable gap between retailers that are customer-centric and data-driven versus and more traditional merchants. Using data and, in turn, analytics to support, inform and drive the business is essential and a core element of being customer-centric. Like Amazon.

Retailers have under-invested in technology to leverage data — especially beyond systems to support merchandise information — and have been lagging in areas like relationship and loyalty marketing. While there are plenty of (generally weak) retail loyalty programs, the key value of such programs is the data yield and the output is a better customer experience. Like Amazon.

Until retail leadership makes the customer a priority in the overall business plan, leveraging analytics, moving beyond mass marketing and creating more valuable customer relationships to drive the business won’t happen.

Brian Kelly
1 year 4 months ago

In my experience, the challenge with internal data is operational silos. This results in multiple views of the shopper. Further, many retailers eschewed D2C marketing therefore their customer insights are incomplete.
For bigger retailers, who have the capex to invest in IT (e.g., Target) they posses the insights and act upon them.
Keep in mind, its not a silver bullet. SHC invested heavily in IT and does posses a superior view of the shopper. Data does not mean insight or even actionable insight.

As all CIOs say, “retail ain’t for sissies!”

Tom Redd

After being in the data and analytics business for many, many years we have seen many situations where retailers WANT the data and want it fast. When that finally took place the data and information were available real-time and there were groups that jumped on it and others still ran on gut feel.

Many of the top retailers in the world that are targeting Amazon vs. being targeted by them live by real-time, instant access to inventory positions, channel action, trends and social activity. They have found that the data science element is not as critical as the press thought and that the user interface is what makes data the better weapon.

We are seeing more retailers adopting the data analysis tools in centralized, single versions with access across all key departments and regions. This is mid-market and large enterprise.

Amazing ideas make amazing retailers and data is the core of amazing retailing.

Camille P. Schuster, PhD.

All industries are struggling. Some retailers have done an outstanding job with analytics related to logistics and replenishment. Some retailers are experimenting with innovative loyalty programs. Some retailers are sharing data with partners to develop products, services and/or experiences for joint consumers. None are doing all of these things.

Analytics is a broad term including logistics, consumer data, pricing, product development, pricing and assortment. No retailers excel in all areas but this is a journey that needs dedication to one area at a time every year to stay competitive.

Mohamed Amer
It’s dangerous to paint an entire industry into a corner; retailers don’t operate on a single speed or mode, yet at the risk of doing just that, let me point to a more fundamental problem that may shed some light on today’s discussion. What tends to be more true than not is that silos continue to not only exist but to thrive in organizations. The role and turf division across the retailing enterprise: merchandising, store operations, and marketing remains alive and well. Each will take their own view of reality and by extension develop their own approach to analytics and their user experience. It’s not only the tug of war between centralized and decentralized operations but defining where that line ought to be for strategy and long-term investment in the business. Lacking strong, engaged, and determined leadership at the top, each of the silos will “run with the ball” as they see fit. That’s not inherently good or bad, but with the fast pace of change in this digital economy, having an inconsistent or nonexistent… Read more »
Peter Charness

The challenge in adapting to being a data-driven enterprise relates somewhat to the Yin and the Yang of retail, art and science so to speak. Data is plentiful, “context” to that data is illusive at best. Some retailers take pride in their success in understanding the customer, and applying “merchant sense” to the market, whether that is gut feel, or fashion sense, this non data driven intuition is a necessary ingredient for success. On the other side of the table is the pure data driven analytics; scientists sometimes end up with an answer in search of a problem. Success requires putting these two often polar opposite skills and approaches together in a single business. Data and context — easy to say, really, really hard to combine for success.

Lyle Bunn (Ph.D. Hon)
Lyle Bunn (Ph.D. Hon)
Strategy Architect – Digital Place-based Media
1 year 4 months ago

Change management is at the heart of data-driven approaches and analytics, where more data than ever before is now available and competitive pressures are so demanding.

Nobody likes to be wrong and analytics can show that some actions should not be taken. Meanwhile, data can indicate that new or different action are required, for which expertise or time is limited.

Analytics take strong leadership and even better soldiers.

Shep Hyken

The problem with using data is that many companies/retailers get too much data and don’t know what to do. They either get “analysis paralysis” or make a mistake by using it the wrong way. But data is powerful, and the best retailers know how to use data. They use Big Data to spot trends. They use micro-data to individualize the experience for individual customers. I’ve had the wonderful opportunity to attend conferences focused on data analytics (Sounds boring, but it is not!) and learned how the best retailers are using data to grow revenue and improve the CX. It’s nothing short of fascinating — how they acquire it and how they use it.

Ken Morris
With the abundance of data available to retailers, real-time analytics represents a prime opportunity to improve decision making and enhance the customer experience. While there is no argument that many retailers have been slow to adopt and implement advanced analytics to improve their operations. As indicated above, innovative retailers, like Amazon, are leveraging analytics to transform the customer experience. Amazon leverages the data about online transactions because they have it available. Traditional retailers do 10%-20% of their sales online and are missing the analytic data from their brick & mortar operations which is 80%-90%+. Until traditional store-based transactions move to the cloud and become part of the real time retail infrastructure, they will be at a disadvantage. This is raising the bar for the retail industry. The good news is that retailers realize they need to leverage analytics to be successful. According to a BRP special report on the Top 10 Merchandise Planning Priorities for 2016, 58% of retailers indicated analytics as their top planning priority. Retailers are finding that many of their current planning… Read more »
Mark Price
In my experience working in retail customer analytics, I have found that the biggest challenges have nothing to do with analytics at all. Rather, the hurdles involve organizational structure, culture and compensation. Accepting insight from a team outside the merchant group is often seen as a loss of power and influence to a group that has long believed that they run the company. The “gut instinct” and “I know the category better than any data could show” culture leads retailers to continue old merchandise selection and discounting practices. For example, every segmentation I have done has shown that some stores index higher for specific segments than others — yet how many retailers do you know that vary store layout and assortment based on that insight? Finally, compensation. If merchants are compensated on margin percent or simply on margin from their category alone, they will be unwilling to adjust products, offers or timing to meet customer needs. Instead they will drive the company to continue to “force” products down the throats of consumers who do not… Read more »
Naomi K. Shapiro
The writing, that is to say, the data has been on the wall for quite a while. In other words (not my own): retailers are “awash in a sea of data” and don’t know what to do with it. Retail’s transition to data-driven enterprises has been arduous for many reasons: The data at first glance, if available, seems daunting and retailers don’t understand how to break it down or focus on the parts that are most important to them; Resistance to mixing the correct analytics procedures with already-existing procedures; Lack of professionals on staff, in fact, lack of analysts on both sides who know what’s going on, what to do with the data and how to do it; Lack of buy-in by decision makers who don’t understand the value and necessity of maximizing use of the data; Lack of accepting analytics as science, which it is: Taking real data, analyzing it for patterns from behavior past, and predicting and optimizing for the future. And yes, data analytics is essential, nay, life-dependent for competing in today’s… Read more »
Doug Garnett

There is a fundamental weakness in this data argument: nearly all the data expected to be used is observed behavior data.

Using this type of data is fraught with extraordinary danger — in marketing I’ve found it to result in Rorschach data analysis. That’s the analysis where the data scientist or manager projects their own beliefs into the data in order to fill the gap of what’s entirely missing in this data: the why behind the behavior.

These leaps of faith filling in they “why” are dangerous. Because very often the data merely reveals some very tiny things. But managers and data scientists are expected to find big things — so they invent the whys and overstate whats in the data.

Retailers need to be cautious about the data fad.

Herb Sorensen
First, I note that the instant poll here does not ask even a single question about shopper behavior, probably because there is a misguided assumption that the queried points are the major factors of shopper behavior — NOT! Secondly, I have spent the past 16 years intensely focused on generating data about how shoppers, you know, actually behave in stores. That is, move, act, purchase, etc. THIS is shopper science, which is driving Amazon to the head of the pack, and bricks retailers are largely oblivious to even what the issue is. Data scientists are of limited value in a field largely devoid of the actual data about shopper behavior in the store, that is relevant to sales and profits. The ignorance is so abysmal that until at least one retailer engages REAL shopper behavior data, they are sitting ducks for whatever competitor first gets a clue, that Amazon’s growing assault is NOT about the internet, but their relation to the shoppers. And as I explain in Chapter 3 of the second edition of Inside… Read more »
Verlin Youd
Some great observations already, so I’ll try to add constructively to the discussion. Risking over simplification, I believe it comes down to the old adage that any successful business solution, including analytics, needs the right combination of people, process, and technology — quite often in that order. People – Retailers need the right executive vision and alignment throughout the organization to truly leverage analytics to compete. Retailers need to break down the oft mentioned organizational silos and fiefdoms required to use a common view of the truth, including the analytics. Retailers need to invest in the skills, both hiring and internal development, and have the patience to see value come to fruition – not expecting miracles after 90 days. Process – Retailers have struggled for years to solve the challenge of master data. Analytics are just as difficult if not more so. Retailers need to define a process to gather data, apply analytics appropriately, acquire both data and analytics where they don’t have the ability to generate it themselves. Retailers need to recognize that they… Read more »
Melanie Nuce

Inventory visibility should be a key component of any discussion about leveraging data analytics. As we have seen with retailers who fully embrace analytics and excel when it comes to implementing them, their success is nothing without the ability to make successful matches between what the consumer wants and what they have available to sell to them. The most innovative, data-driven retailers realize that item-level RFID is the key to knowing what is in-stock, where it’s located and having confidence that inventory levels are accurate.

"Using analytics to drive key business decisions may very well define the winners and losers of retail. The bar has been raised."
"There simply are too many choices and the pitches by these companies are too complicated to understand their differences and their value."
"We are seeing more retailers adopting the data analysis tools in centralized, single versions with access across all key departments and regions."

Take Our Instant Poll

What are the most critical activities retailers need to analyze?

View Results

Loading ... Loading ...