How to draw a Big Data implementation roadmap

"Big Data implies that data sets are so large and complex they become awkward to work with using standard tools and techniques."

That was the definition of Big Data given by Rod Bodkin, founder and CEO of the consultancy, Think Big, at the 2014 Teradata Partners Conference taking place this week in Nashville, TN.

Buried within this simple definition lies many problems and pitfalls that retailers and other organizations face when their data needs exceed their current capabilities. In fact, Mr. Bodkin was quick to point out that new, affordable technology has enabled the growth of shopper and product data though new customer touch points and new retailer channels.

But organizations often struggle with how to best get from where they are today to a happier place where their database(s) and their access tools meet the requirements of doing business competitively.

Migrating to a Big Data solution is a journey, according to Mr. Bodkin, and the trip is rife with pitfalls and challenges. Among them are:

  • Sorting through the corporate politics of siloed databases and siloed applications;
  • Attaining a vision for the enterprise under a Big Data platform;
  • Avoiding damaging short cuts for the sake of time or expense savings;
  • Addressing skill gaps among current personnel;
  • Assuming that the journey will be "business as usual."

To the contrary, Mr. Bodkin asserts that Big Data inherently involves change management and must be sponsored at the highest levels of the organization. To be successful in attaining corporate goals and financial rationale for the effort, it must be approached holistically, not as an ad hoc project that is layered upon the many other initiatives the organization has on its docket.

To mitigate at least some of the pitfalls and enhance the chance for successful implementation identifying "low hanging fruit" — in other words, a pain point that a Big Data solution can remedy that has an existing, accepted ROI — is an excellent starting point. From there a "roadmap" can be drawn that shows incremental milestones, which should be met and communicated as often as every 45 days. Finally, along the way assess the need for enhancing skill sets and adjust accordingly. In many cases, the team you have today will need to augment their expertise and perspectives for success.

BrainTrust

Discussion Questions

What “low hanging fruit” should retailers focus on as part of the Big Data implementation process? What steps do you think are being underestimated by retailers in efforts to capitalize on Big Data?

Poll

10 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Dr. Stephen Needel
Dr. Stephen Needel
9 years ago

The underestimation is that it’s inherently worthwhile to try and capitalize on Big Data. There’s an attitude that because it exists, it must be useful—and that’s not always true. Creating a process to use Big Data is backwards. The data needs to be used to solve problems/answer questions, not to have a life of its own.

Zel Bianco
Zel Bianco
9 years ago

Too numerous to post but one that can be significant is to organize and perhaps purge those data sets that are not leading to action. Combine what you have internally as a retailer with the more progressive suppliers who are bringing you valuable unstructured data and build the roadmap together. You will get to the holy grail faster than trying to go on the journey alone!

Paula Rosenblum
Paula Rosenblum
9 years ago

I just have a different perspective on this. Definitely retailers will ultimately want to capitalize on the data that’s coming in from paths to purchase and social media, but I think there’s a more basic first step: Capitalizing on the data they already have.

We are loaded with SKU/store or channel/day data—both sales and receipts. Aggregating this data into meaningful, actionable information used to be pretty much impossible, or the requirements of science data analysts. Now, computing power has enabled us to actually use the data we have in far more powerful ways.

So my own vote is stop thinking about “Big Data” as some abstraction, and get pragmatic. What would you have LIKED to do with your existing data that you couldn’t in the past, and can advances in hardware and databases make them possible today?

Beyond that, we have some demand/supply analysis to do, as we do more distributed order management, but that can come a bit later. In the meanwhile, it’s a good idea to get your arms around what you’ve already got, and what you’d like to do with it.

Gajendra Ratnavel
Gajendra Ratnavel
9 years ago

Big Data in retail?

Never mind the snail’s pace at which retail seems to upgrade in-store technology. We’re still waiting on line-busting tablets and automated wait-lists for out of stock items! Their back-end systems are ancient.

I walked into the Calvin Klein underwear store yesterday. The sales agent said they didn’t have my size. OK sure, when will you get them? “Sorry, no idea.” Can you call someone? “NOPE.” When do you get stock? “Oh, it’s random.” This last one shocked me. Usually I get, “Come back Tuesday, we may get new stock on the item you want.”

Point is, retail needs to connect their back-end technology to “collect” the data first before you can mine it for useful information.

I am very skeptical this situation will be improved. The starting point for Big Data is enhancing the database systems — getting in-store technology to improve the number of data points collected, for example, such as traffic counters, automated surveys, interactive terminals in store, automated heat mapping, etc.

Online retail is in much better shape to take advantage of Big Data.

Ralph Jacobson
Ralph Jacobson
9 years ago

There is a perspective that “data can be your secret weapon, but using it effectively is much more about processes and people than about tools,” as writers Joerg Niessing and James Walker state. There is no question that there are effective tools available in the market today to enable decision making. However, the people and processes must be taken into account in order to optimize decision-making capability. That’s where the value of big data and analytics lies.

I think CPG and retail companies have a huge opportunity to leverage the effective capture of “clean” data. That is, ensuring the data to be analyzed is actually trustworthy. From there, companies should definitely evaluate the tools out there for their purposes, but be certain to not underestimate the internal cultural challenges that most certainly exist that will become obstacles to effective decision-making once the data is analyzed.

Matthew Keylock
Matthew Keylock
9 years ago

My tip would be: don’t start adding new data until you are already doing a good job with existing data.

Cindy Christian
Cindy Christian
9 years ago

When you get into big data, you realize that there’s a lot of it, and it certainly can be overwhelming. As others (Stephen, Paula and Matthew) have already mentioned above, the best approach is to focus on solving your business problem (vs. wrangling the data) and to use the data you already have available. But even when you are able to look specifically at a current set of data, e.g. POS, there are many more possibilities that come to mind than one can (or should) practically pursue.

Today, the challenge facing most retailers is “Where do I start with big data?” The key is finding the best “low-hanging fruit” and opportunities with the biggest bang.

We have just released a new Brick Meets Click Big Data Survey that is designed to help with this challenge. The survey evaluates 12 specific areas of big data application in customer and merchandising strategies. We invite you to take the survey and add your thoughts to this important conversation—here’s the link.

Bill Bittner
Bill Bittner
9 years ago

Big Data is just a popular term for a new analytical tool. The only thing the business should need to understand is the capabilities and limitations of the tool. The mathematical complexities and impact on IT databases should be completely hidden.

Like any major project, the place to begin is with a baseline. Retailers are constantly forecasting their requirements, whether it is how many cashiers to have on site between 5:00 PM and 7:00 PM or how much inventory to buy for the coming spring. Forecasts are necessary when planning organic growth and especially when considering acquisitions. Big data has the potential to improve all these predictions. Gather baseline results that show what the variations were between forecast and actual results.

Now it is time to put the data to work. The fundamental point is that you are no longer going to be trying to answer the “why” question. Causation is no longer the question; if the data improves the forecast, we don’t really care why, we are simply going to use it. That is important for a couple reasons. Improved data includes everything from sophisticated classification of what you already have to installing new sensors and capturing data you never before collected. It will undoubtedly turn out that there are only a few types of data necessary to get significantly improved results. There is no need to capture and save data that does not improve results.

Mark Price
Mark Price
9 years ago

We define Big Data as any data set that is too big for the organization to easily deal with. By this definition, some of our clients struggle with dealing with integration of e-commerce, customer service, retail transaction and product data into a single database that can be mined for opportunities.

The lowest hanging fruit tends to be cross-sell and retention efforts for high value customers.

Retailers underestimate consistently the time and resources necessary to build such databases, when documentation and data quality of the source systems may be scarce. In addition, they underestimate the level of consistency and focus necessary to bring such activities to reality and stay with them to drive ROI.

Seeta Hariharan
Seeta Hariharan
9 years ago

Yes, I agree that the migration to a Big Data solution is a journey. One of the first steps is to understand the data maturity of the enterprise. Conceivably, you can define four levels of data maturity:

At level 1, enterprises use data merely to run their business.

At level 2, instead of just running their business, enterprises use data to manage their business.

At level 3, enterprises start to use data to enable innovation.

At level 4, which is the ultimate level of maturity, enterprises clearly view and use data as a competitive differentiator.

As organizations transition from level 1 data maturity to level 4, it helps them move from being just reactive to getting predictive and more importantly, becoming prescriptive.

As Mr. Bodkin says, Big Data initiatives must be sponsored at the highest level, but to truly succeed, they must also permeate every aspect of the enterprise.