Will a data scientist shortage hurt Big Data’s promise?

Discussion
Apr 30, 2015

Through a special arrangement, presented here for discussion is a summary of a current article from the blog of LiftPoint Consulting.

Data scientists are a new breed on the marketing team, with expertise across a diverse collection of areas, each of which can have a positive impact on marketing initiatives, separate and combined.

Yet the McKinsey Global Institute predicted that "by 2018 the United States could face a shortage of between 140,000 to 190,000 people with deep analytical skills, as well as a shortage of 1.5 million managers and analysts who know how to use the analysis of Big Data to make effective decisions."

A survey by Robert Half Technology concurs, suggesting that "most companies aren’t maximizing their data collection and don’t have the people in place to do so."

These hard-to-find people can be your tour guides through the complexities of data. They combine computer science, statistics, math, and business skills with creative problem solving and clear communication to help you make marketing sense of all that data.

In recognition of their value, marketing and analytics are already merging in the C-suite, with CMOs increasingly aligned with their CIOs. In 2012, only 36 percent of CMOs said their CIO was a critical partner; by 2014, that percentage grew to 51 percent, according to "The Evolved CMO In 2014," a joint research project by Forrester Research and Heidrick & Struggles.

Data scientist shortage

There is urgency too. Forrester Analytics’ State of Customer Analytics 2014 report concluded that analytics is no longer an option, but a necessity for any organization to compete. Every organization in every industry needs a senior-level data specialist on their marketing team.

Data scientists can translate all the analytic mumbo jumbo into concepts and theories that non-analytic marketers can understand. They recognize that a given business question exists inside the context of a given company and industry, and the nuances those outside influences play on the technical work of solving the business problem.

Many firms outsource their data scientist needs if they can’t afford or can’t find the correct skill set. The benefit that an external data scientist brings is a perspective on marketing from outside your company ("best practices") that can be leveraged to save time and improve results for your team.

The days of marketing as a "creative" fraternity are over. Today’s marketers need a data translator to help question, discover, interpret and, ultimately, succeed in today’s data world. The era of data scientists has arrived.

How important is it for retailers and brands to leverage data scientists? What are the barriers that can prevent a data scientist from succeeding on a retail marketing team?

Braintrust
"For what it’s worth our data shows the exact same thing — a shortage of talent. I think this puts pressure on the vendor community to provide easy-to-consume data. They can hire the talent and make it easier to use."
"There are plenty of vendors that will slice and dice a retailer’s data and provide reports ad nauseam. The problem comes when the retailer tries to use the data to improve and change outcomes. That’s where that special person who, as Mark points out, "can translate all the analytic mumbo jumbo into concepts and theories that non-analytic marketers can understand" shines."
"We must be careful here. As Mark and Marge point out, the person needed to handle this data is not just a data person. Maybe not even a data person."

Join the Discussion!

14 Comments on "Will a data scientist shortage hurt Big Data’s promise?"

Notify of

Sort by:   newest | oldest | most voted
Paula Rosenblum
Guest
2 years 7 months ago

For what it’s worth our data shows the exact same thing — a shortage of talent. I think this puts pressure on the vendor community to provide easy-to-consume data. They can hire the talent and make it easier to use.

Marge Laney
Guest
2 years 7 months ago
There are plenty of vendors that will slice and dice a retailer’s data and provide reports ad nauseam. The problem comes when the retailer tries to use the data to improve and change outcomes. That’s where that special person who, as Mark points out, “can translate all the analytic mumbo jumbo into concepts and theories that non-analytic marketers can understand” shines. I would argue that the person described in the article is not a pure data scientist. Data scientists focus on the details of data sets and methodology. The person Mark describes lives between the data scientist and the non-data… Read more »
Zel Bianco
Guest
2 years 7 months ago

I don’t think anyone would question that analysis is necessary, and as the article states (and we can all see as we go about our daily lives) most data is not being fully utilized. Data scientists are essential, as are tools and education to help non-analytic marketers and business users better understand and interact with the information available.

Gordon Arnold
Guest
2 years 7 months ago
It is my opinion that big data is a runaway nightmare. The problem is redundancy caused by a never-ending supply of incompatible data file structures that must be polled separately for information reporting purposes. An example of this is demonstrated by many of the internet search engines that report identical information offerings/statistics and count them as separate or new. Some search engines have filters that are automatic or manual used to reduce multiple reports of a single event and the confusion users are left with. This process only adds to the need for more support and system resources. The proposition… Read more »
Cathy Hotka
Guest
2 years 7 months ago

I participated in a Wharton School of Business conference this week, and the importance of data science was a common theme.

There’s a huge disconnect in this country. We’ve gold-plated our colleges and now 80 percent of students need financial assistance. We need to make it easier for our students to to acquire the skills necessary to make money in the knowledge economy.

Vahe Katros
Guest
2 years 7 months ago

At first I thought of the movie The Imitation Game and imagined retail offices taking on a look like Bletchley Park, then I thought of a near future with an abundance of unemployed data scientist-lites who heard this article’s call to action only to be later displaced by improved tools, then I wondered if McKinsey used data science to construct this “click-bait” scenario. Then I realized that I was in trouble and took my medication. Do I get the job?

Gene Detroyer
Guest
2 years 7 months ago

We must be careful here. As Mark and Marge point out, the person needed to handle this data is not just a data person. Maybe not even a data person. We are educated to analyze, but handling big data by analysis will just get you bogged down in the trees and you will miss the entire forest. Big data is a picture of a dynamic system at work.

The person who will see it the best will be a synthesizer, not an analyzer. They are hard to come by.

Graeme McVie
Guest
Graeme McVie
2 years 7 months ago
A shortage of data scientists and managers who can use the analytical outputs will hold back retailers from realizing their full potential. I’ve been working with retailers for the past 15 years to deploy advanced analytics to help with decision making. Going into most engagements most retailers said their biggest concern was whether the analytics would work in their businesses. There was never any doubt that the analytics would work but my biggest concern was the ability of the organization to digest the outputs and use them on a consistent basis to make more analytics-driven decisions. There’s a big difference… Read more »
Bryan Pearson
Guest
2 years 7 months ago
The role of Chief Data Officer might become one of the most important positions in the c-suite. Gartner predicts 25 percent of all large global companies will have someone dedicated to the role of data oversight by next year. Here are five facts Gartner shares about these CDOs: 1) The role is growing to the power of 10: More than 100 chief data officers—that specific job title—work at large organizations today. That’s more than double the number Gartner counted in 2012. 2) Finance and government lead the way: The role of the chief data officer is most prevalent among banking,… Read more »
James Tenser
Guest
2 years 7 months ago
There’s a special breed of folks out there who can’t necessarily do the math, but who can interpret findings and explain their significance to others. Marketing decision-makers need people like these on their teams, I’d argue, even more than they need pure data scientists. Many Big Data analytics are—or will soon be—pre-packaged. Database design innovation and new analytic tools are arriving thick and fast. Expertise in how to design the query will be increasingly at a premium in this era. That takes business acumen, supplemented by an understanding of data technology. The new crop of Marketing MBAs can certainly write… Read more »
Lee Kent
Guest
2 years 7 months ago
Yes, retail needs data analysts, but may not so much data “scientists.” An entry level data manipulator who knows how to smash and mash data but more importantly, the wise old sage who knows how to match said results to the corporate mission. If you have read Malcolm Gladwell’s “Blink,” you know what I’m talking about. Success is in finding those 4 or 5 key “realities” that will drive your business. Let me add that I am not talking about using data to personalize marketing messages. That is just mapping. I am talking about using data to drive the business.… Read more »
Bill Davis
Guest
2 years 7 months ago

Very important and yes, this will adversely impact the retailers and brands that don’t secure qualified personnel. The three biggest barriers to be overcome, IMO, are:

1) senior management making decisions because of what they think as opposed to what’s really happening
2) being able to change how things operate today as well as establishing and adhering to a roadmap for how to achieve the desired future state
3) finding a balance between data, intuition and common sense

Retailers will have to address each of these three items, and likely more, before data science is more widely accepted in driving business decisions.

Kai Clarke
Guest
2 years 7 months ago

The McKinsey Global Institute? How many studies does this data embrace? What about other resources? The nebulous information and data referenced here is skeptical at best. Cannot believe these conclusions from dubious positions.

Michael Day
Guest
2 years 7 months ago
Big data’s promise has a bit of its sheen taken away by the absence of not just data scientists, but really any other agency that is not available in full supply to take advantage of it. Data scientists, in theory, have the skill sets, the knowledge and the experience to agglomerate data from a variety of sources and of many types, cleanse the data and then apply a combination of analytics techniques to gain insights. It takes a special blend of characteristics to do even a passably decent job, let alone a stellar one, to derive these insights. When data… Read more »
wpDiscuz
Braintrust
"For what it’s worth our data shows the exact same thing — a shortage of talent. I think this puts pressure on the vendor community to provide easy-to-consume data. They can hire the talent and make it easier to use."
"There are plenty of vendors that will slice and dice a retailer’s data and provide reports ad nauseam. The problem comes when the retailer tries to use the data to improve and change outcomes. That’s where that special person who, as Mark points out, "can translate all the analytic mumbo jumbo into concepts and theories that non-analytic marketers can understand" shines."
"We must be careful here. As Mark and Marge point out, the person needed to handle this data is not just a data person. Maybe not even a data person."

Take Our Instant Poll

What is the most common reason marketers are slow in bringing data scientists onto their teams?

View Results

Loading ... Loading ...