[Image of: RetailWire Logo and Tagline (for print)]

When Will Big Data's Reality Meet Its Vision?

February 20, 2013

The following article originally appeared on the SymphonyIRI Group CPG Blog.

Seemingly everywhere you go in the retailing and CPG industries today, it's about personalization, big data and having the right product for the right person, every time.

Yet, for many consumers, the reality is not the same as the vision. Consumers appreciate customized offers and personalized service, but too often retailers fail to complete the circle they are now capable of drawing with better data and improved analytics. Here are a couple of examples:

1. A top tier sporting goods retailer - A popular sports product retailer sends out e-mails regularly to subscribers, touting specials, as many retailers do. But its e-mail program is not smart enough to know that the chain does not carry a particular size in a brand of shoes (the same size as this e-mail recipient). There is a way for the customer to figure this out, but it requires a little sleuthing. In this case, contacting the company via e-mail from its site elicited no response, so the shopper tried its corporate Facebook page. That got a response from a company employee, but it was a lower-level associate who said it was an odd size, and that it was the manufacturer's fault anyway. There were plenty of inappropriate comments on this particular employee's Facebook page, as well. In an age when Zappos carries seemingly every shoe and size, plus offers free shipping both ways to boot, brick-and-mortar retailers need to better link their loyalty and e-mail programs to better target e-mail offerings.

2. A top tier drug chain - This one is real simple. In some cases, when swiping a credit card, the customer is asked to provide their zip code to the employee. This is what I call "reverse personalization" because it is presumably being done to make sure the credit card is not stolen. In any event, in one recent case an associate was given a zip code. After entering it, he announced, "That is not your zip code!" But it was. When requiring additional information from customers, associates should be trained to explain to the customer in a friendly fashion why the information is required and why it is in the best interest of the customer to provide it.

In short, there are now many streams of data available to retailers, but the training of associates on how to use it, how to talk to customers about it and how to personalize online and offline shopping experiences doesn't seem to have kept pace. It's as if we are constantly layering on new data streams before we have the basics right. So the customer often ends up with a disjointed shopping experience that makes one wonder if the retailer has any idea what things look like from the shopper's perspective or if the merchant is just trying to jam as many sales through the funnel as possible. This leaves a large opening for retailers that care and spend a little more time catering to the wants and needs of individuals.

I'm reminded of a corporate cafeteria I frequented while working in an office complex some years ago. They had a special loyalty club for employees of the corporation, but allowed non-employees to use the cafeteria, albeit paying increased prices. After a few months of hearing grumbling about this, the cafeteria came up with a special program for non-employees to use. They got a little better deal than someone off the street, but not as good as the employees. It was called "The Outsider's Club."

Discussion Questions:

What has your experience been with the use of big data to personalize offers and experiences? Who is doing it right and training their workforce to use these new data streams appropriately?

While we value unfettered opinion, we urge you to show respect and courtesy for people or companies about whom you comment. Keep in mind that this is a public, professional business discussion. RetailWire reserves the right to edit or refuse the publication of remarks that we deem unsuitable. We may also correct for unintended spelling and grammatical errors.

Instant Poll:

How close are retailers to delivering the benefits promised from the insights associated with Big Data analysis?


Progress in this area is evident, but clearly there is a long journey ahead for most retailers in term of using data for customer recognition and benefit. I have always struggled with the concept of overtly recognizing VIP shoppers in the store, being concerned about how the average customer might feel seeing someone else getting better treatment then they do. However, we are in an age and time where relevance is such an imperative with the shopper, I think the benefits of such recognition now outweighs any downside.

But having access to shopper data in multiple silos, managed by multiple departments is a recipe for disaster if the retailer is truly interested in developing a holistic relationship with the shopper. One view of the shopper is needed along with a customer strategy that guides the communication flow and dialogue with shoppers. Finally, it is key that whatever this customer strategy entails, the associates in the store must have access to accurate data and be trailed "religiously" as to how to use it to interact with shoppers in order to execute that strategy.

While this journey is long for many retailers, there is no doubt in my mind that it will return a huge return, financially and otherwise!

[Image of: View Braintrust Panelist button]
Mark Heckman, Principal, Mark Heckman Consulting

The digitization of all our available shopper touchpoints is generating massive amounts of data at an exponential rate. While there is no doubt that there are incredible insights and benefits to be garnered from this data, it is all undermined if the insights and benefits are not designed, created or valued from the shopper's and customer's perspective. All too often the promise of the data and the technology that captures the data is valued like a rough diamond—imagine what this will be transformed into and how valuable it will be!

Those brands and retailers that filter and create direct benefits and experiences that are valued by their customers from this torrent of data will win the 'share of wallet' metric. Big Data should take a back seat to quality data. If retailers and brands cannot turn these marketing and merchandising ideas into 'closed-loop' processes then they're probably not worth doing.

[Image of: View Braintrust Panelist button]
Adrian Weidmann, Principal, StoreStream Metrics, LLC

Spire, dunnhumby and EYC come to mind when I think about delivering the right offer to the right person at the right time. The key is finding the golden nugget(s) in all that data. As I used to say with clients while CEO of Kenosia. "It is more important to understand and do something with small bytes of data than be proud of collecting all the data and not knowing what to do next." These 3 solutions help retailers and brands figure out the nuggets.

Some new interesting solutions are also coming to market. Whether they can make it long term is yet to be determined. One that comes to mind is FreeMonee.com. Great solution to reward consumers with something you know they want (thanks to credit card purchase behavior).

Big data and how it is used still has plenty of room to grow.

John Boccuzzi, Jr., Managing Partner, Boccuzzi, LLC

I find the over use of the term "Big Data" to describe customer interaction quite comical. "Big Data" has been around for decades.

In the '90s, Oracle coined the acronym VLDB (Very Large Database) to describe massive data volumes. That acronym was picked up by all the database vendors and used to describe data warehouses, backup/recovery systems and replication databases.

"Big Data" has been around a long time. Understanding how to design the system so that "Big Data" provides value is both a technology issue as well as reporting and end-user issue. First, the data has to be automated, clean, reliable and in a format that allows for easy access by end users. From there, CRM (Customer Relationship Management) takes over. On the end-user side, CRM can be either transactional and provided immediately, or analytical.

However, the technologies used are different because CRM designed to run the business is different than CRM solutions designed to manage the business. Products like Siebel are commonly used by companies to provide immediate information to the cashier, teller or other user about the customer in front of them. CRM solutions such as Cognos, BlueSky Analytics Business Objects, are analytical CRM solutions designed to help users understand the masses and make strategic management decisions to significantly improve profits.

"Big Data" is just that... massive data volumes. Organizing it and formatting it so it is accessible is what will allow end users to provide good customer service. Providing that good customer service is being able to leverage that data to improve customer relationships. But it also involves a positive personality, good attitude and proper training on the tools used to understand your customer and products.

[Image of: View Braintrust Panelist button]
Janet Dorenkott, VP & Co-owner, Relational Solutions, Inc.

The scenarios described in this discussion are caused by the lack of awareness of how many and what kind of search fields are needed to implement the program successfully. If the program is designed and built to be aware of needed information parameters and their potential locations in the data files, the sort would include only complete files. The first few attempts to populate the advertisement program can easily be reviewed for feasibility and potential against planned fulfillment of financial return requirements.

Sending a message to consumers that have needs beyond the product availability is an error in the report design and build, not the amount of information sorted as in big data. The companies wishing to design and develop a marketing and sales software capability need to understand that this is a process that must be managed, tested, documented and implemented with rigid controls.

Most of the hundreds of company IT enterprise systems that I've visited and tested for security, disaster recovery and business continuity were seriously lacking, simply by not implementing and supporting an effective corporate IT oversight committee. Oversight committees should never include management or executive membership other than as observers. These committees should report actual ease of use and support capabilities with a log of consumer feedback information excluding any and all subjective insight. Repopulating the committee on a quarterly basis will serve to provide an influx of new perspectives and issues insight.

When only a few people from a few departments have any and all to say about the program the odds of failure increase exponentially when time and money limitations are added to the needs of the company. Blaming the hardware for its inability to address the companies' needs is an excuse only surpassed by blaming the size and quality of data.


Big data is a relatively a new tool for retailers. As correctly mentioned, most retailers are nowhere close to the vision. A lot of groundwork has to go in to first identify the sources of data, set up infrastructure, capture the data in the required format, and then make sense of it all. Most retailers today are just embarking on their journey. With the IT systems spaghetti they have today, getting to the data itself is a huge challenge.

Retailers who have embarked on this journey are looking to first get this data in one place and then make sense of it all. Sears, for one, has achieved this milestone to getting it all in one place and now working on how to make sense of it.

For customers to see a true 360 personalization, that will have to wait a little longer. However, spurts of excellence can be experienced every now and then, when the retailers get their insight to action cycle right.

[Image of: View Braintrust Panelist button]
Shilpa Rao, Practice Head - Merchandising, Tata Consultancy Services

Of course, the vast majority of retailers and CPGs have not captured all the relevant insights from their flood of data in the ecosystem...yet. There are some great examples of companies leveraging the data to drive better shopper insights, though.

One retailer began exploring the improvement and enhancement of its already powerful web site to continue the focus on customer retention. At the same time, they observed that its customer base was increasingly pressed for time and demanded high degrees of speed and simplicity. These factors taken together pointed to the need to deliver a much more personalized shopping experience to its online customers to live up to its brand promise.

This retailer is now testing some tools to give its marketers the ability to drive different messaging and click behavior to customers according to customer attributes. These attributes can range from whether or not they are a first-time user of the site, to which zip code they reside in, to past purchasing behavior. Marketers can bring these variables into a campaign utilizing these tools, all without the intervention of IT staff. The ad spot will then display different content to different customers based on who they are and what customer segment they belong to.

Training their marketing workforce eliminates the expense of IT spending time analyzing the data.

[Image of: View Braintrust Panelist button]
Ralph Jacobson, Global Retail Industry Analytics Marketing Executive, IBM

Relevancy, Your Honor? Maybe it's just me but I fail to see the role or application of Big Data in any of the above examples. Is knowing a customer's shoe size or matching a cardholder's Zip Code on record to the in-store inquiry considered Big Data?

I'm not saying that these aren't interesting anecdotes, but I'm disappointed that this post doesn't address the topic or impart any information or insight regarding the headline per se. Just trying to help the community stay focused and sharing insight vs. stories....

Jim Koppenhaver, Associate Partner, IBM

I personally do not believe that Big Data is new to retail. Having been in this industry for 35+ years, capturing and handling data have always been on the forefront. I remember when the 'database' came to be, but I digress.

The issue here is not the amount of data, but how and what pieces of it you use and then the consistency that you deliver it with across all channels. The employee issues, that seem to be the real problems here, point out how important it is that all customer facing activities must be consistent and communicated across all channels too.

Lee Kent, Sharing Insights for Success in Retail, YourRetailAuthority

Lots of very good comments here and I think a common thread is that it's not the availability of data that's the issue: it's beginning the collection process with some notion of what you want to do with it that is where most retailers fall down.

I recently signed up for a large drug retailer's new loyalty program. Because I'm a pharmacy customer, this chain obviously has a lot of my personal information but as I signed up at the main checkout and started clicking through screens, my birth date flashed up on the POS terminal. ACK!!!!!

Yes, intellectually I know they know that information, but that's not really something I want to see connected to my loyalty program and my purchase of cashews and a diet coke (full disclosure: there might have been some chocolate in there too). As shoppers we choose who we want to do business with based on notions (sometimes misguided) of trust. The cognitive dissonance described here in the shopping experience undermines that trust. Al is right: simply explaining why you are asking for a piece of information or what the advantage is to me in giving it will go a long way. Having a plan for what you (the retailer) plan to do with a piece of data is a critical first step. As many have outlined here, it can be hard to access and action it so tread wisely on what you collect.

Lisa Bradner, Chief Strategy Officer, Geomentum/Shopper Sciences

It takes personal selling skills, along with big data, to keep this from simply becoming more SPAM in the marketplace.

[Image of: View Braintrust Panelist button]
Herb Sorensen, Ph.D., Scientific Advisor Kantar Retail; Adjunct Ehrenberg-Bass, Shopper Scientist LLC

My experience has been horrible because "BIG DATA" really doesn't know what I want, so I just keep getting bombarded with offers that I am not interested in receiving. Big Data has become junk mail on steroids and I can't see that it is doing retailers any good! I can't find anyone who is doing it right but Publix—who isn't "doing it" at all. If you want a tailor—made suit, you have to go to a tailor; you can't get one at Penney's.

As for data streams, their aren't any new data streams that matter. These new data streams are just noise that causes merchants to "jump to conclusions" based on a consumer's whim. Just because I look at an insurance policy doesn't mean I want 13 emails, a call from 6 agents—give me a break BIG DATA!!!

If you want to train your workforce, there are a thousand things they need to know before they learn anything about big data and the consumer. Teach them to speak well, make change, stock the shelves and to properly order stock. If retailers are concerned about anything, it should be OOS. They lose more money because they don't have inventory than they could ever make by trying to use BIG DATA to personalize a sales pitch to me.

Ed Dennis, Sales, Dennis Enterprises

Great story.

It seems that Big Data efforts are being primarily directed toward the consumer to understand them and bring them to the product on the shelf. Some very significant tools have emerged in that effort. If you have not had a review with MyWebGrocer recently, you owe it to yourself to do so.

I watched a major new product introduction in a fairly large chain store over a 4 week period. It was missing from the shelf on the first Tuesday I visited the store, and was still missing this last Tuesday. Tag was there, as well as the sign touting the new item...just no inventory to buy.

Think about the POS sales data running into the demand systems of both the retailer and the vendor, the tools shuffling expectant customers to that empty shelf...probably never to return, and the value of that roughly 6 square feet of empty shelf space.

Perhaps it is time to turn the power of Big Data to the sales growth trapped on shelf conditions poorly reported through old-school measurements, POS data, and self reported sales force efforts. Bill Bishop is preparing a thorough review of Big Data, and what it SHOULD be in his BrickMeetsClick blog.

Mike Spindler, Managing Partner, Panther Mountain Companies, LLC

A great example of using big data the RIGHT way is with STM in Montréal. Okay, a longer story, but amazing how they are using this massive amount of rider data.

Here are the basics. The Société de Transport de Montréal (STM) is the largest public transit corporation in Québec and it supports more than 2.5 million riders every day with its bus, subway, and other rapid transit services. STM, in fact, was named best public transit system in North America in 2010.

STM wants to continue to increase marketing efficiency, optimize the effectiveness of its promotions, and create direct relationships with the consumer. This requires big data support and fast analysis and is the foundation of their goal to increase its ridership—with a loyalty program targeted at the one million passengers who use their OPUS smart card to pay for transit.

The smart cards store customer information that includes for example purchase points. STM will analyze this information to better understand different customer segments and individual travel habits. For passengers participating in the loyalty program, STM will use this insight to support personalized interaction with the customer over their mobile phones. STM can push relevant notifications or provide tailored information about local shopping or upcoming area events. STM has about 1500 partners in retail, events, restaurants and other transportation companies such as taxis to help it reward STM ridership with real-time, money-saving offers.

By rewarding loyal passengers, STM hopes to increase ridership. Customers will receive an even better value for their money, and retail partners will benefit from additional advertising.

All of this is not possible without a strong big data foundation. Big Data is making big changes in many ways—from travel, to shopping, to loyalty.

[Image of: View Braintrust Panelist button]
Tom Redd, Global Vice President, Strategic Communications, SAP Global Retail Business Unit

Big Data. Aren't we talking about bringing in the next level of Business Intelligence to an organization? Industry has coined a new term that we call Big Data. Every retailer goes through a phase of transformation their IT landscape to meet the growing data need/demand (input/output) of business.

However, on the way to transform the IT Infrastructure, not much of focus was given to an analytic process/tool that will take care of all of the data (an organization possess) within a time which would help business to make quick decision (may be real time).

This issue started getting addressed in the recent past with advancement of technologies like Virtualization, Cloud (IaaS/PaaS/SaaS), Advanced & Fastest Analytic tools (e.g. Hadoop), etc.

I would say the focus/strategies of retailers could be to have a single cloud infrastructure architecture (even with different data centers/servers) and then having those advanced analytic tools over the cloud architecture, to bring next level of business intelligence to the organization. This may be just the first step to start getting the initial benefit out of Big Data.

Aditya Ranjan Samantara, Senior Business Analyst, Accenture

There are known problem domains (like physics and genomics) or companies (with huge daily transactional bread crumbs (Amazon, Walmart, Facebook) that can effectively implement long tail interventions (read: relevant and interesting offers).

For everyone else, the battle is clear—it's John Henry vs. the Steam Powered Hammer. It's intuition vs. the machine. What to do?

Implement your existing network of pre-frontal cortex computers (PFCCs). Also known as 'a few smart people connected together collaboratively over skype', if provided the right kinds of 'why and how' data, they can see around corners and create new opportunities.

These types of systems require a new corporate operating system (aka culture), but for firms that don't have the breadcrumbs to drive meaningful interventions, it might be the way to go. Coffee is not optional.

Vahe Katros, Consultant, Plan B

Big Data is just that (which is of course relative, but that is a topic for another day). Big Data by itself does not make smart marketing strategies and it certainly does not train sales associates. What Big Data does is enable smart marketing strategies and allow well trained sales associates to better service and communicate with customers. However, endemic for retailers is to be transaction oriented and not customer oriented. This will lead to the first problem outlined above which blasts customers with an email regardless of whether or not it applies to the target. It will hit some of the right target and it will drive some transactions and may be considered a success even if it alienates some customers.

Big Data, or any sized data for that matter, is an enabler and marketers needs to start thinking it as a means to an end, not the end.

Kurt Seemar, President, Analytic Marketing Innovations

Just look to Amazon.com for lessons in using data to personalize a customer's experience. The key is getting data on the customer that is relevant. Then using that data to properly promote and engage them with their specific needs, wants, etc.

Customers are afraid to give their information unless they, as the old saying goes, know, like and trust you. Once you achieve trust, you can get all kinds of important and relevant information that will be helpful in building an even stronger relationship with the customer.

[Image of: View Braintrust Panelist button]
Shep Hyken, Chief Amazement Officer, Shepard Presentations, LLC

Search RetailWire
Follow Us...
[Image of:  Twitter Icon] [Image of:  Facebook Icon] [Image of:  LinkedIn Icon] [Image of:  RSS Icon]

Getting Started video!

View this quick tutorial and learn all the essentials...

RetailWire Newsletters