NRF: Why Are Retailers Still Complaining About Big Data?
We hear the complaints about all the data that is now being captured from TLogs in the store and online leading to "a plethora of data but a dearth of wisdom." After walking the floor at NRF, I am confused why anyone would say this. I was amazed at the number of BI (business intelligence) vendors ready to help retailers cope with the volume of data they are collecting. Representatives from Teradata, Aldata, 1010data, and Micro Strategies were among the ones I saw or spoke with about this complaint. Further, ARTS (The Association for Retail Technology Standards) has created an RFP for BI solutions that describes the features necessary in a solution. Yet the complaint persists. I have come to the conclusion that there are plenty of viable solutions; the challenge for retailers is knowing what to want from the data.
BI tools open a whole new opportunity for gauging business performance. More importantly, new message-oriented interfaces make it possible to provide real or near-real time performance information for store operations. This combination of detail and timeliness makes it possible to provide feedback while there is still time to correct a bad situation. For example, knowing that sales on a promoted item have stopped is a sure way to warn people in the store that the shelf is empty. On the other hand, seeing that an item has higher sales than expected may lead to a special offer on another size of the same product. Targeted markdowns by store allow control of individual store inventories based on actual sales so that overstocks and returns are minimized. TPIs (tactical performance indicators) can be developed from the data that are focused on job performance and provide employees feedback on their efforts. TPIs also highlight the advantage of equipment upgrades or an improved business process.
The use of BI to evaluate consumer behavior, however, is still more art than science. But even here the retailer is striving for only three possibilities: the consumer buys a higher gross item; the consumer shops a department they have not previously shopped; or the consumer buys more of an item they are already using. Switching the consumer to a different item based on BI clearly depends on the category; few people will switch to private label cigarettes, for instance. But the BI tools should be able to tell in which categories the consumer buys multiple brands and enable the retailer to suggest better items.
The propagation of BI solutions raises another question: who really should be influencing consumer item selection; the brand product manager or the retailer? Retailers would naturally want to shift their customers to the higher gross private label items — probably not what the brand manager wants.
Discussion Questions
Discussion Questions: With tools apparently available, why are retailers still complaining about the difficulty in extracting wisdom from Big Data? Do retailers know what they want from the data? How are you seeing BI tools changing the influence of brands at retail?
I’m not sure who is complaining about Big Data other than the processing and analyzing of data has always been a formidable task for most retailers given their general aversion to investing in talent and systems to get the most out of the data they generate.
But the larger problem of data overload that Bill Bittner speaks to is real. Having dealt with these issues first hand, I can also assert that without a C-level strategist who is empowered to effect pricing, merchandising, marketing, and even advertising, data analysis and resulting applications are destined to be more ad hoc than strategic and thus lead to inconsistent results.
Once a customer data plan is in place, focus can be trained on specific categories, customer segments and even stores, that is necessary to target a limited content to the areas of highest return. Having a plan and strategy also will be a positive signal to CPG brands who are looking for retail partners they can trust long pull to use data to drive their mutual interests.
“New news” seems to be breaking everyday relating to retailers investing in long term contracts with NCR, EYC, Inmar, Valassis, and others that put the retailer in an enhanced position to finally start harvesting meaningful and action based analyses. But the key to success in harnessing data remains firmly planted at the retailer’s feet. It is their responsibility to commit to the process with empowered, strategic leadership.
Having supplied brands and retailers with recommendations and guidance based upon quantitative data and analytics, I’ve found that they are drawn to the absolute nature of data but often lack the commitment and cultural collaboration to respond to what the data is telling them. People want ‘actionable data’ but seem to be reticent to actually respond to the required action that the insights are suggesting. The insights often require a cross-functional (departmental) response and that becomes difficult to actually execute.
There is certainly an interest to value data and BI that supports an existing agenda or initiative, but when the insights suggest a different compass heading, the head winds that would be encountered to facilitate a change are often too difficult to overcome. Retailers tend to be focused on operational issues and look for the ‘quick fix’ on existing processes and workflows rather than designing innovative (and often disrupting) strategies based upon quantitative data and customer/business intelligence.
We are not in the business of generating or selling data. We are in the business of helping people figure out what to do with data — and that does not mean “tools.”
Bill hits the nail on the head. Databases are like dungeons. If you don’t know what you are looking for before you go in there and don’t have a torch to light your way, you will never come out.
Clearly the problem isn’t technology. And frankly this isn’t “big data” either, in fact in the grand scheme of things it’s rather miniscule. “Big data” is what geneticists are going through when they have to wait for over a year for a search of their data to be completed because it’s so massive.
The most dangerous part of a swimming pool isn’t the deep end. It’s the narrow section between the shallow and the deep where you don’t know whether to stand or swim. That, it seems to me is where retail is when it comes to data and perhaps technology as a whole.
‘Back in the day’ a retailer could ‘feel’ what was happening, when and how to change an approach to a customer, what products would work and so on. To do that retail had to be in your blood. Now we’ve complicated the customer experience to such a degree (most of our ‘insights’ we just made up and have little validation for) that we’re looking to technology to make real sense of it all for us. Can it even do that?
Now may I ask a question of the team here? Is there some kind of central depository of what we might call “retail data?” Or does “data” refer to what might be happening in a specific retail entity — Sears, Costco, etc. — and it’s all located in 10,000 places? Please respond in this venue or, if you prefer, please contact me directly through RW to discuss. Thanks.
Data mining and needing a true use for the information, let alone knowing how to use the information, are the key issues. At the heart of all of this is that retailers are capturing data, but truly have no use for much of the information. Data mining is a solution looking for a problem. Retailers KNOW that they have to prevent OOSs, give better customer service, and keep their stores clean, clear and easy to navigate for their customers. Training their staff, and maintaining low prices are the other side of the coin (sometimes). However, these key basics are still not addressed enough, let alone what to do with the data that their POS system captures every minute of every day. The expertise at retail is the customer’s retail experience, NOT how to mine data to squeeze every bit of information about a particular product, category or time period. This is why Data Mining and use of this information continues to be ignored….
What a timely discussion after a great NRF show in NYC this week. I started working with retailer data (grocery) 15+ years ago and even then retailers — and frankly, manufacturers — were complaining about too much data. It’s not about the data and how much or little you have, it’s about the questions you are laser focused on answering that count. People still think that if you collect all the data and run it through a BI solution, a magical answer will come out the other side that will make you more successful.
I sat through a presentation on Tuesday presented by the CFO of The Vitamin Shoppe and the President and CEO of 4R Systems. In one hour they took us through their process of dramatically improving store performance through the use of data. They started with a laser focused question and worked backwards from there, using the data available to create power answers that they then acted on and benefited from. It’s not necessarily that 4R Systems has some magical solution, it’s more about how the two groups worked together to understand the business issue, research and then react to what the research showed.
Thankfully the industry has dozens of powerful BI solutions available at several price points, so almost everyone in the industry can afford a solution. It’s not about size of the data or the BI solution you use, it’s about the laser-focused questions you are looking to answer and more importantly, knowing what you could do to improve the business if you knew the answer.
The powers that be in retail intuitively want big data and all that it brings. However, the reality is that they can’t execute against it. It tells them that the way they have been operating (from their gut) for decades is wrong. Big data is going to tell them to change the way they address customer acquisition, the way they handle inventory and the way they project their sales.
The challenge is not to collect data (retailers have plenty of data to mine) nor even to find a software to make sense of it. Retailers must work in partnership with their software vendors to go beyond a laundry list of features and benefits and produce the reports that really matter to them. What matters the most to retailers is neither the quantity of data nor the software features, but the access to actionable reports that retailers could use as coaching tools. Very few software vendors actually take their clients from viewing to doing.
Big data tend to be anecdotal talking points among people who are not in the trenches. After doing some work in the trenches in retail, I realize that “small data” is very important.
I believe many BI companies are trying to bring slow corporate BI practices to fluid environments such as retailing. Retailers are complaining because KPIs and dashboards do nothing for them when the weather forecast is predicting rain for the next several days.
Retailing needs real-time data in small chunks at all levels, not past 12 year forecasts and trends. I would argue from my experience, retailers need more “predictive” data that can monitor real-time data and detect and respond to patterns in real time.
Bill: Great insights, and I think you hit the nail on the head: retailers are complaining about big data because they don’t know what they want from it. Big data won’t help you if you don’t have a strategy already.
The other problem is the age-old “in-house” syndrome. Too many retailers want to manage these massive data sets internally, when they are clearly beyond internal capabilities.
Big data is clearly the future of successful retail, but effective use will require new skills, a willingness to outsource, and a clear strategy.
Know your path and follow it. Capture info and it goes into a repository that neither has the logic in place, nor the tools to empower the merchandising and/or the marketing side of the house, and all you are doing is creating “big data” with no clue. Consultants who have a process for asking the right questions to lead the enterprise near- and long-term can help.
Customer intelligence at work is a mantra to follow — if it’s collected, there better be a plan on why you are collecting it and what you are going to do with it. Old habits die hard and some on the merchandise planning and allocation side of the house have their long-standing methodologies to drive their decisions, or they haven’t seen a dependable/usable solution to support a change.
Lots of vendors selling solutions to overcome this complaint. What are they missing? My impression is that it is a lack of strategic/tactical application informed from the data. The tools are smart, but as shared you need in-house resources or hire the right group of consultants to lead and empower the retailer for decisions, communications, purchasing and display strategies/tactics that are winners on a consistent basis.
Okay, I sort of think we’re missing the point. The value of “big data” is getting actionable insights at a granular level QUICKLY into the hands of decision-makers (NOT score-keepers). That has never been possible before. Moore’s law, wide data pipes to stores, and mobility has helped make that possible.
So what retailers need may be the same, but the speed at which they can get it has changed – dramatically. That’s what big data is all about. It’s a short-cut for “actionable insights at granular levels delivered in near-real time into the hands of decision-makers.”
That’s a very new story, not an old one at all.
Retailers who complain about the challenges in translating data into actionable insights are simply not prioritizing. It’s now easier and less expensive than ever to manage Big Data but retailers consistently under invest and shift priorities.
That said, it’s still not easy as it takes people, tools and discipline that is generally not resident within most retail organizations. The good news is that this is changing and every day we see more retailers investing in this area. Whether it’s through an innovation group or marketing or even IT, it’s happening and will continue to happen at better retail organizations.
It’s very clear if you read the quarterly earnings reports and listen to the conference calls, the retailers that understand Big Data and focus on customers, best exemplified by Amazon, are the ones that are thriving and putting the others out of business.
Across industries, executives are swamped with data. Many companies extract the needed information. The big problem is the look and feel of what is reported. It’s up to the buyer and user to determine what they need and how best to report it for action. Accepting the typical template and dashboard can lead to feeling bombarded rather than supported. Business intelligence can only be activated when those examining it can easily determine what they need to know. A little up front planning as to how to organize and show the output can make the difference between data and insights.
Heck, retailers have never been able to manage data, even before they didn’t have too much to manage. The inability of retailers to manage data resulted in laws being passed in the 50s and 60s that fined retailers heavily for running out of merchandise that was advertised via a sales flyer or radio/TV. This law was lifted in the 80s after some very heavy and expensive lobbying.
The fact is that retailers often just don’t have a clue. I have two local grocers (chains). Every year during the Christmas Season, one of these retailers runs out of stock on holiday baking supplies, the other does not. This has happened every year for the last ten years. Now the OOS retailer has been bankrupt twice and has just recently been sold, so I expect the problem to disappear. However, that will ultimately depend on who in the old organization the new owner retains. But this multimillion dollar retailer had tons of supply chain information and consumer complaints, but chose not to act on the information. If you know you are losing business and making customers mad and won’t act on it, then why would that retailer ever complain about having too much information?
Tools are an enabler, but are not the whole solution, and in fact may in some cases make life harder for the retailer.
There are broader organization fundamentals that need to be in place to be successful and many are at odds with the way retailers are used to working. If you are interested in some perspectives on these, here they are…
(1) A clear strategy that the organization is truly behind (many retailers run on tactics and financial metrics). This allows the business to know what direction to head in, which questions to ask and how to apply the answer consistently across the business over time.
(2) An organizational design and readiness that can use and apply the insight. For instance, people with the right skills; organized to transcend the politics; processes that incorporate the use of the answers from the data; and incentives that focus on the right behaviors and insights (not just short term, one-dimensional product or category focused financials).
(3) The capabilities to turn the raw data into useful data. It’s very easy to get summaries, segmentation or insight wrong and this can be very dangerous as I’ve frequently seen it provide the opposite recommendation to the right one. Technology companies can’t do this well and rarely can retailers either. This is a rare skill, but critical to success.
(4) The approach for managing data and software for insight and customer-based action-planning is very different to the approaches needed for managing data to keep the business operational. People, skills, processes, requirements gathering, and technology “best practices” are different. However, in most cases the hardware, software and data are managed by the same IT organization in the same way under the same performance metrics.
Unless a retailer (and manufacturer for that matter) is ready to tackle these, it is unlikely to achieve much benefit beyond tactical and operational improvements.
My experience is skewed in FMCG, especially grocery, which I’ll speak for here. Let’s first describe “Big Data” from a retailer’s perspective. I suggest it is the detailed transaction history from their POS systems, for those with online channels, online transactions included. There continues to be intense retailer interest, due to proven returns, in shopper centric marketing. Derivations of transaction data inform assortment, pricing, promotion, and advertising decisions with increasing localization and also support highly relevant individual customer communication.
Software “tools,” including BI systems, are only enablers of this movement. Frankly, many BI platforms are not especially good at the necessary transactional analytics. The central issue is the effective application of the tools, within a retailer that is fully committed to a shopper marketing culture.
Imagine bringing your sick automobile to the dealer and being given an engine analyzer and a chest full of tools with, perhaps, some encouraging words. This is the plight many retail exec’s feel they face.
The Big Data problem is solved when domain knowledgeable expertise is applied with domain-specific software tools against the right strategic retail objectives. An example most readers are aware of is dunnhumby’s work with Kroger. In their case, a huge deployment of retail-experienced human capital has been applied, using home-grown software tools. The results are compelling. I don’t think there are any commercial BI tools in the mix. Some would argue that SAS (used there in modeling) is a form of BI, but that’s splitting hairs. In harnessing Big Data to drive retail results, tools alone are not solutions, and generalized software systems are not always the best tools available for the job.
This one is simple. The grocery industry as a whole has inadequately invested in systems and personnel since the first round of scanners hit stores. Many are currently saddled with legacy systems that need to remain in place for purely financial reasons.
To make matters worse, we are now living through a period where a popular strategy is to delay capital investments. So, instead of moving into this decade’s opportunities to use new technologies in more forward looking applications, many companies are content to wait just a bit longer. Heck, we don’t even really forecast in this industry — we scenario plan. Not too many math research degrees in the halls of grocery companies.
Data is a wonderful thing; it can speak to us with great nuance, but you have to be able to extract from that data relevant information and actionable knowledge, and then you have to be able to execute on that knowledge. I don’t think many retailers can really do either, and I think they know it.
The challenge with extracting relevant information and actionable knowledge is that what’s relevant is always shifting and moving. This is where human experience and judgment is critical. In order to get the right answer you have to know what the right question is. It’s not necessarily pre-programmable. Further, the questions that senior retail executives are primarily interested in is how can I reduce my costs. I don’t really think they see data as a way of increasing sales, they see it as a way of containing costs and keeping up with the competition. As a result, they simply aren’t investing in the skill sets necessary to consistently convert data into relevant information and actionable knowledge.
That leads to the inability to execute at store level. The data extraction that is pre-programmable, and can be converted into relevant information and actionable knowledge, is stymied by all of the issues we discuss so frequently here regarding store level employees. Store level employees are seen as another cost to be contained, not as a critical resource to be invested in. Retailer executives don’t view increasing the store level skill sets as something that will deliver a return on the investment. So, they knowingly accept that the ability to execute at store level will be less than it could be, despite all their other efforts.
The answer is simple, but execution is always the challenge. In order to leverage the data and transform it into something actionable, a retailer needs to do one of two things: 1. Hire the skilled personnel who understand the business and the tools needed to extract actionable information, or 2. Hire a partner who can do the same.
The problem is that the data so far outweighs intuition and ‘innate retail intelligence’ (i.e. gut) within retail orgs now that most people reading the data don’t know what to do with it. As in any research, there’s always a qualitative factor…why are the numbers saying that? What should I ask these numbers for? Etc. Who would know that now? The CEO that was a CFO?
Goes back to another retail dilemma; are we hiring the right people? Are we hiring too many scientists and too few merchants? I personally think, yes, the latter is true.
There’s a big difference between data and knowledge. Data are necessary to take store management out of the realm of guesswork and into that of true, fact-based decision making. But data points on their own are highly difficult to work with, especially when we’re talking about the complex, nuanced environments we’re seeking to optimize.
Our customers have achieved great success by using a solution that takes the huge quantity of data available and converts them into views and reports that yield actionable insight. Our solution measures more than 9000 individual data points on the average store visit. Nobody wants to examine 9000 data points per visit. Instead they want to see correlations, comparisons, and models that help them understand the true behavior in the store. They want to see heat maps of traffic and how those maps change over time. They want to see the cyclicality of the day, week, and year and how that compares to staff schedule. They want to see how all of these things correspond to actual sales at the register. And they want to be instantly alerted to certain conditions like when a certain item is near empty on the shelf or too many people are standing in line. That’s how Big Data can be useful, when it’s translated into actionable insights that make shopping better.
The article missed the point. What Teradata, Micro Strategy, etc, offer as analytics solutions have been around for some time. The advent of Big Data is also not new, albeit with a different name — item level data, RFID data etc., Application logs have been in existence for ages. The existing analytics apps as-they-are cannot address these volumes of data, their asynchronicity and demand for low latency.