Data analyst

Do retail marketers have an appetite for data science?

Due to a lack of skills, marketers spend a good portion of their budgets outsourcing analytics, according to a recent Gartner report.

The report states, “A notable share of the analytics budget — more than technology and nearly as much as internal talent — goes to outside experts. The majority of mature data-driven marketers expect external sourcing to grow over the next two years, and 30 percent of them expect to decrease their internal team size, taking more advantage of the efficiency, scale and expertise of service providers.”

This is surprising. Many of these same companies are striving to become data driven and building analytics into their corporate DNA. According to recent Mckinsey research, the top ten U.S. retailers now have an average of almost 70 big data analysts on board.

Consider too that companies realizing business value from Big Data are typically not buying off the shelf services and pre-defined models. They are doing the work themselves to create competitive differentiation.

Somewhat astonishingly, Gartner finds “concerns such as ongoing costs, data security, black-box models and lack of transparency, and loss of competitive advantage are all valid reasons organizations may choose not to use external service providers. However, for most marketers, these considerations will not deter investment.”

There seems to be a disconnect.

Either marketing doesn’t believe their data scientists have the right skills or marketing isn’t aware of the talent available because data scientists are consumed with non-strategic aspects of the job.

Another possibility is that marketing doesn’t believe that their data scientists are able to execute projects in a timely way. This could be because of inefficiencies in the analytics process itself.

“Smartsourcing” is a good way to think about resolving the disconnect:

  • Acquire the analytical talent on-demand to support a use case your team cannot handle, and/or;
  • Acquire a Big Data cloud capability to account for portions of the analytics process you lack or which takes too much time.

Clearly there is an opportunity to do something different. According to June 2015 CMO Council research, “Lack of measurement can hold marketers back and could be a reason why many are not adopting omnichannel strategies at all.”

BrainTrust

"The answer doesn't seem to be either/or as much as 'and.'"

Ryan Mathews

Founder, CEO, Black Monk Consulting


"Analytics ownership needs to remain in-house. However, analytics and data science are and should not be limited to the marketing department for use."

Roger Saunders

Global Managing Director, Prosper Business Development


"Retailers need to find the blend of internal and external that will give them that competitive edge."

Larry Negrich

Director, SaaS Marketing, Zebra Technologies


Discussion Questions

DISCUSSION QUESTIONS: Do you think marketers are wise to outsource so much of their analytics? Do you think internal data science teams are up to the challenge of supporting their companies’ needs? Does smartsourcing sound like a good idea?

Poll

23 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Mark Price
Member
7 years ago

Retailers today face a difficult challenge in their analytic strategies. They need to move quickly to identify and take advantage of market opportunities, and may need to leverage an analytic infrastructure that could take months if not years to develop, particularly when you take into account the integration of multiple sources into a single view of the customer.

Hiring data scientists does not solve this problem. The best analytics is not valuable to a company if the insights cannot be acted upon. Outsourcing analytics to a firm that combines data science with a deep understanding of what is practical can permit retailers to accelerate the implementation of the insights dramatically. Over time, retailers will become more adept at conducting their own analytics, however if the goal is to accelerate time to action an outsourced firm represents a strong opportunity.

Note: my firm provides customer analytics and execution support for retailers, so I am slightly biased :-)!

Doug Garnett
Active Member
7 years ago

Marketers should be developing the expertise in-house. But I find there’s a tendency in data science to believe sophisticated and complex analysis is the key to success (and that’s certainly the magic pixie dust these outside firms sell).

I’ve found that simpler analyses executed by people closer to the problems tend to discover more useful insights. In-house teams understand better what insights lead to concrete steps. And in-house teams understand which insights will have the biggest benefit to the organization.

Tom Redd
Tom Redd
7 years ago

Before the data science hype a retailer must focus on the data resources.The “what” element of this whole push for science and retail is the data. Having Big Data is not the only way to solve this. To move from WHAT to HOW and act on data science efforts, retailers need to create a data plan that addresses the data sources — from TLOG files (sales audit data) to inventory details, trends, etc.

From there it is having a resource where it all can be stored, kept fresh and updated fast. After that foundation or platform is in place the science dance can begin. Without the data foundation the science is all a major guess.

Lee Kent
Lee Kent
Member
7 years ago

Retailers in the olden days held everything they did close to the vest for fear of giving away the family secrets. Over time they have come to learn that if it isn’t a core business practice, then maybe they can use the same ERP everyone else does, or use SaaS, BPO.

The key is determining who the brand is and what are the key components. I don’t know about you but, for me, who the customer is, is key! And this, IMHO, is the family jewel.

While I get that retail may not yet have seasoned data analysts, young, bright talent coupled with veterans are a good place to start. I agree with Doug. Methinks we over-complicate analytics. Have you read “Blink” by Malcolm Gladwell? That’s a good place to start.

And that’s my 2 cents

Ryan Mathews
Trusted Member
7 years ago

I’m not so sure in certain retail channels — supermarketing for example — that you can make the case that data analytics have helped all that much in the first place. True, supermarketers have leveraged data to do a more efficient job of managing a dwindling share of market, but was that really the goal?

So I think we should start by drawing a distinction between customer-focused analytics and organizational-facing analytics.

Stop! I don’t want to hear the argument that all analytics in the end benefit the customer. They do, in a semi-abstract way, but often not in ways the customer notices.

As to whether the data should be contracted or done in-house, it’s easy to say that contracting gives no competitive advantage and therefore homegrown systems are better. To that argument I can only say, “Can you spell Kmart?”

We live in a world of increasingly open-sourced analytics and one where the pace of technological disruption challenges the notion that DIY analytics are best.

That said, I see no reason why the internal analytics teams shouldn’t be as good — or even better — than the teams they contract with.

So, bottom line, the answer doesn’t seem to be either/or as much as “and.”

Ross Ely
Ross Ely
7 years ago

Data science is a complex field that is evolving rapidly, and retailers are wise to rely on external resources to provide this function. With external resources, retailers can take advantage of best-in-class capabilities on a pay-as-you-go basis. The external provider can tailor a solution to the retailer’s environment while still leveraging its functional expertise in data science. Retailers should strive to learn and understand analytics but continue to use external resources to provide the function until the technology has matured.

Anne Howe
Anne Howe
Member
7 years ago

Big Data can be used in so many ways that enable quick actions, but I too believe that analytics deliver more when coupled with internal human insight. To do that really well, retail marketers need to commit to hiring on both the quantitative and qualitative sides of analytics.

Just imagine if you could discern that shoppers who were identified as emotionally attached to the retail brand (let’s say 30 percent of all) were spending at triple the rate of the other 70 percent. That would cause some definite action plan adjustments, right?

But if we never bother to identify an emotional linkage, we can run data all day long and never learn which of our shoppers help us on the road to profitability.

Charles Whiteman
Charles Whiteman
7 years ago

“Is smartsourcing a good idea?” sounds like a leading question.

Joking aside, when it comes to strategically important work, it’s always a good idea to be thoughtful and really understand what your partners are doing for you.

With a vendor, you can often structure a pay-for-performance arrangement that isn’t possible with full-time staff. Data scientists are in high demand these days — meaning it’s expensive to hire and retain them.

Smartsourcing is a good way for companies to get real results quickly. Insisting on vendor transparency allows these companies to retain the knowledge and insight that comes from this work.

Kim Garretson
Kim Garretson
7 years ago

I am skeptical about all the money being spent on data scientists and outside analytics services. One reason is that another word for “predictive” in analytics is “guessing” (See excerpt and link to recent Venture Beat article below). With retail marketers now adopting better first-party data by simply asking consumers for permission to market to them on their own criteria, I think combining this data with predictive data could be a way for retailers to better utilize their data assets.

From VentureBeat: “It turns out that marketers are spending well over a third of their budgets (on average) on analytics. This in spite of the report finding that levels of confidence in analytics’ ability to generate insight are mediocre, at best.”

Cathy Hotka
Trusted Member
7 years ago

Retailers will happily tell you that they’re behind in terms of data science. Most retail companies have multiple analytics engines, usually siloed by department, and no clear vision of what they’d like to do with the oceans of data points they maintain. It would be a huge step forward if the industry could define what successful analysis could ultimately do.

Adrian Weidmann
Member
7 years ago

Since I provide quantitative data analytics services as part of my business, I am delighted that retailers and brands alike use contracted services. I believe it is a question of agility and focus. As a small business owner I am able (and willing!) to react very quickly to specific requests. Additionally, having been a pioneer in the evolution of digital signage, in-store shopper behavior, media and experiential design — my analytics engagements leverage this experience as I understand the context in which these particular data points exist making it easier for me to help the client gain the specific insight they’re interested in revealing for themselves, or on behalf of their customer.

A marketer at a retailer and/or brand may have a specific mandate of business objective that needs to be supported by quantitative insights. It is easier, quicker and cheaper to contract an outside resource with specific and unique expertise within the context of that business objective than trying to prioritize and reallocate internal resources that are focused on big-picture initiatives.

It is not a question of either/or but rather of how internal and external data science resources can, and should, complement each other to provide invaluable insights that translate into a better shopping experience for your customers.

Paula Rosenblum
Noble Member
7 years ago

It’s easy to say we should develop the expertise in-house, but our data tells us that data analysts are hard to find.

Let’s face it. They can make a lot more money in the financial sector. Why come to retail?

Ben Ball
Member
7 years ago

“Can you spell Kmart?” Ha! That one brought a chuckle and recollection of one of the biggest donnybrooks I ever saw between a client IT executive and a third-party provider (us). The argument was over database integration of an acquisition. IT VP said, “piece of cake — no help needed.” My guy said, “spell Kmart.” To my amazement we still got the job, including the database integration task.

But to the question — using outsourced data analytics cripples internal marketing teams in a very dangerous way. Whole generations of marketers are reliant on the single interpretation of the data the outside analyst gives them because they don’t know how to use the source data or how to deconstruct the “Final Report” they receive to look for other possible conclusions. They are held hostage to one conclusion as gospel when any decent analyst knows there are at least several more that could be equally valid. Give me a data set and tell me what you want it to say and I can get the data to say it. If I have an opinion and you don’t, then my opinion is gospel when I report the “results” of the data. But it’s not even a question of objectivity really. I’m not saying the outsourced analysts are intentionally misleading clients. It is just that they are only going to give you ONE answer most of the time and that one answer is usually NOT absolute.

Tom Brown
Tom Brown
7 years ago

Data scientists are a rare commodity. The good ones are extremely rare.

Camille P. Schuster, PhD.
Member
7 years ago

Great marketing analytics require two skills: understanding what questions need to be asked and being able to use appropriate tools. These skills often require several people working together. These are not skills usually required of people in the marketing department of the past. These are, however, the skills needed now for analytics to have an impact. The first skill is difficult to outsource and will determine what kind of analytics are needed. This is difficult to outsource unless you believe that understanding your consumers requires the same generic questions as every other company. It may be necessary to outsource some of the analytics because it is difficult to hire one or two people who are excellent at using every tool and are able to link the result back to business issues. Control of this process should stay within the company if you have employees who know the right questions to ask.

Roger Saunders
7 years ago

Analytics ownership needs to remain in-house. Marketing is a logical and practical space to drive the discipline, as it focuses first and foremost on the consumer. However, analytics and data science are not and should not be limited to the marketing department for use.

Executive, financial, merchandising, and store operations teams should exercise their “walking around” skills to express how and what they might be seeking from data science. There is a breakdown in this arena, as too often analytics, which are relatively new to retail (past 10 years), are seen as black box specialists. They’re not. They reinforce the common sense that all associates need to embrace.

Retailers can find vendors who can assist along the way in building the internal bridge to better data science usage. Vendors, be they CPG, service or manufacturers, have a strong interest in helping retailers grow their businesses. That is why many of them have already started the process of building their data science expertise.

Find the support. Build the data science from within.

Peter Charness
Trusted Member
7 years ago

I think there are (at least) two scenarios. Firstly, “We know what we are doing and what we are looking for, but we need to buy more bandwidth of skilled expertise to get the job done,” and secondly, “we have no idea how to approach this Big Data idea and would just like someone else to come back with all the answers.”

I think retailers should strive to be in the first camp. Have enough experience in-house to create the vision, and properly manage the oversight of an outsource team. I think caveat emptor needs to apply to a retailer who presumes to have the expertise, only to select someone else who will potentially lead them down a garden path of Big Data benefits.

Ralph Jacobson
Member
7 years ago

Great post, Gib! Without succumbing to my inherent career prejudice, I can say in today’s marketplace, in the U.S. and around the world, retailers are wise to leverage tools available to take the burden off of data scientists on staff. Of course, internal organizations can manage some of the requirements of their companies’ needs. However, why would you rely solely upon internal staff when technologies can literally replace multiple data scientists more effectively for less expense?

As in most things in life, I believe there is a balance to be reached in every retail organization. Internal management needs to look at their core strengths and true data analytics needs and consult with multiple external experts to help identify the best path forward.

Larry Negrich
7 years ago

Marketing is all about the results and superior analytics brings a competitive advantage. Retailers need to find the blend of internal and external that will give them that competitive edge. Internal can give retailers the ability to review deeper, more customization, and pivot quickly. However, outsourcing can bring best-practices, delivery of new capabilities and perspectives, and freedom from learning curves and resource constraints.

Kai Clarke
Kai Clarke
Active Member
7 years ago

Marketers should be outsourcing their analytics. As we know, Samuel Clemens said it best: “There are lies, damn lies and statistics.” We should focus on our KPI strengths, and retailers should be focusing on their customers. Since history shows us that retailers are classically data challenged (most don’t even manage the data their POS creates), delegating the management of this data, so that creating the reports to inform the retailer better should be left to a data science specialist.

Vahe Katros
Vahe Katros
7 years ago

Retailers have relied on intuition and those ideas can be turned into algorithms — but to do that, we need scarce talent. I guess that’s the bet IBM is making around machine learning. I wonder what kind of online marketing campaigns Watson would suggest if it saw diapers and beer in a shopping basket.

In the mean time, it probably makes sense to uncover the unknown-unknowns as in, what kind of questions can/should I be asking now (think: personalization/profitability/experience improvement) and what kind of questions might I ask if I had the data, and how much does it cost to execute on that? Just thinking about this is giving me a headache! If RetailWire used machine learning, I wonder if they would put up banner advertisements for Advil next to stories like this. Oy!

Naomi K. Shapiro
Naomi K. Shapiro
7 years ago

As a representative of a company that provides Retail Intelligence technology at an extremely high level of sophistication, my answer is an amalgam of my own and other BrainTrust respondents’ answers to today’s question: Do you think marketers are wise to outsource so much of their analytics? Yes. The ideal is to combine sharp analytics teams in-house with outstanding technological capability and experience of the outside vendor who specializes in the data germane to the specific company using the services, and the capability to gather this information.

Do you think internal data science teams are up to the challenge of supporting their companies’ needs?

Retailers are going in this direction, but they still have a long way to go. The most powerful move is to combine the capability of the ones who have the questions that need answering with the capability and experience of seasoned vendors who know where this info resides and how to gather it most effectively for the client’s use.
A question that was raised, but not addressed by anyone re: Retailers afraid of “loss of competitive advantage (as a reason to choose not to use external service providers).”

Answer: They need not worry, at least in our case. I believe I speak for our company when I tell you that we exercise the highest and most absolute and meticulous discretion and separation from client to client. Each client is treated as a discrete unit; the only thing they have in common is using our advanced technological capability, i.e. tools to gather and analyze that data that helps each of them approach and deal with their specific and particular questions and needs.

Finally, there is no substitute for insight, the human ability to be discerning in looking at the data and developing and applying answers and solutions. That is also built into our system — the ultimate responsibility we share with our client is to provide them with the capability from which they deduce answers and solutions.

Finally, instead of taking an aspirin (or maybe a Maalox) per today’s final comments, I’m going to write an immediate blog post dealing with this RetailWire discussion in more detail.

Hi-Tech BPO
Hi-Tech BPO
6 years ago

Great article, Gib!

I just bumped into this amazing piece of work on retail analytics, and also know that it is too late to share my thoughts on it; but couldn’t stop myself from doing it. Views by a lot of dignitaries in the comments section above are commendable and should be considered to reach out to data analytics maturity.

Most of the retailers combat multiple analytics engines, usually siloed by department, and no clear vision of what they’d like to do with the oceans of data points that are collected and remain in silo. From data collection, to data entry, and data processing, categorization and validation; they provide data modeling solutions competent enough to be applied to experimental, social, financial, industrial and geographical types of data. Their proven expertise of empowering retail companies to act on data trends, to attain business goals, in their industry, in their town, with their government policies while adhering to their business rules; are noteworthy.

Their statistical analytics include linear and non-linear regression analysis, logistic regression analysis, multivariate analysis, time series modeling, hypothesis testing and validation, experimental & observational studies etc. Their data, analytics, visualization and business dashboard solutions provide valuable business insights to identify opportunities, improve efficiency, maximize effectiveness and enhance customer experiences.

Don’t mind me saying, but “How to hire such data scientists as part of in-house team, and someone to lead them – will be a new challenge retailers will have to combat”, upon taking the DIY approach.