What’s needed beyond KPIs?
COLLOQUY staff
Through a special arrangement, what follows is a summary of an article from COLLOQUY, provider of loyalty marketing publishing, education and research since 1990.
According to Jonathan Adler, director of insights and analytics at Lenati, KPIs (key performance indicators) are good at providing snapshots, but subpar at providing any context.
“For example, suppose retention was down three percent last month,” Mr. Adler said in an interview. “Is three percent a big drop, or is it the kind of month-to-month fluctuation that should be expected? Was it from customers across all segments or just one type of customer? Was it at one location or across the company? KPIs typically won’t answer these deeper questions.”
While marketers sometimes think having more KPIs will provide more clarity, too many can be overwhelming, and mixed findings from those KPIs can erode trust.
To move beyond KPIs, companies should consider tapping exploratory and predictive analytics.
Exploratory analytics views the data in different ways to get more context of the larger story. Business intelligence tools like Tableau and Excel, or data science programming languages like R or Python, support the deeper analysis.
Establishing a strong relationship with insights and analytics resources is key to exploratory analytics.
“Ideally, you have someone who can translate a business question to a data science question,” said Mr. Adler. “Then go find the right data, and translate that data science answer back to an answer that can be widely understood by the business. Asking your team to explore a data set or respond to a specific question is very different than requesting them to ‘make this graph.’”
Predictive analytics uses advanced statistics and machine learning to gain an understanding of customer lifetime value, customer churn and segmentation. Said Mr. Adler, “For example, suppose there’s a retention problem. You can use predictive analytics to build a statistical model to understand the drivers of customer churn, and predict which customers are likely to leave next.”
Success with both approaches requires a consistent reporting structure — a digital dashboard, a monthly report, an Excel sheet, etc. — to identify issues and find answers quickly.
Discussion Questions
DISCUSSION QUESTIONS: Are marketers overly reliant on KPIs in understanding and leveraging customer data? What advice would you have about leveraging exploratory and predictive analytics?
Poll
BrainTrust
Mark Ryski
Founder, CEO & Author, HeadCount Corporation
Adrian Weidmann
Managing Director, StoreStream Metrics, LLC
KPIs are useful snapshots, and exploratory/predictive analytics have a place too, but this is not an either/or proposition. As someone who has spent more than 20 years analyzing store traffic and conversion data, the issue I see is that there is a dearth of expertise in extracting insights from data and in interpretation. Collecting data is relatively easy and there’s never been more at the disposal of marketers, but producing reams of graphs and charts doesn’t necessarily lead to insights.
In my experience, data interpretation is more akin to meteorology where trends and patterns inform insights and general conclusions about probable outcomes. While sophisticated modeling tools are important and helpful, they need to be used by someone who has the expertise to know how to interpret the output. Rarely are the answers black or white.
KPIs have changed with data availability. Analytics at different levels of abstraction fuel operational, tactical and ultimately strategic decisions toward a healthier corporate valuation. On this path improved brand equity, leverage of assets and improved productivity of places, processes and people are realized. A business that is not data-driven and focused on gaining insights and acting on them is doomed.
Most of the time you don’t need complex analytics to understand KPIs (although that can help). If you have the right KPIs for your business, the snapshot is sufficient. You want to look for changes that are outside the normal boundaries of variation-over-time — this was Deming’s point for as long as I can remember. When a KPI indicates a significant positive or negative trend, then it’s time to dig in and ask why, either to fix it or to leverage it. The answer may be a simple one or a complex one — start with the simple approach.
Apparently we are still devoted to the Newtonian model that organizational systems are simply a collection of parts and if we keep measuring each part something good is bound to happen. Is anyone else shaking their head at a sentence like: “You can use predictive analytics to build a statistical model to understand the drivers of customer churn, and predict which customers are likely to leave next.”
Customers are abandoning us in droves so let’s try to predict who’s going to leave next. What would we do without consultant-speak? Good grief.
The only way to make your organization unassailable is: 1.) Identify the highest possibility the organization wants to make real. Be aware, this is not the typical yawn-producing “mission statement.” 2.) Align every point of energy within the organization and focus that alignment on that possibility. Most organizations have very little or no energetic alignment.
This requires a total change of mindset and thinking-process so it’s not easy. The old adage “What is measured gets done” needs to be replaced with “What is BELIEVED gets done.” The brutal reality is that digital dashboards, monthly reports and Excel sheets aren’t the answer.
KPIs are just a proxy for what’s truly going on in your business. As a result, they are always merely an estimate and too often a weak estimate.
The thing that makes KPIs both useful and misleading is the fact that they oversimplify reality. This makes it critical to avoid managing by the metrics. Rather than being the focus for the way you manage, they should be one item that informs it.
Edwards Deming suggested, in fact, that managing to the metrics leads to the perfectly-managed business … that fails. It’s sadly too true.
It’s also critical to complement KPIs with other input. They should be combined with traditional consumer research (especially qualitative), walking the store, talking with front-line and back-office management and a good sense of the competition.
And this is what makes business fun. Because management can’t be turned over to algorithms — we aren’t slaves to KPIs. Successful business decisions require executives, informed by a wealth of research including KPIs, using their analysis, insights and instincts to make difficult decisions.
This is exactly where retail management needs to look for actionable performance insights today. All too often, the answers you need to make better decision are hiding in a very deep ocean of data. KPIs are still necessary and hold real value, however there needs to be much more information provided to make better decisions. Cognitive-powered solutions help you tap into the extraordinary capabilities to learn, understand and advise. New AI technologies can help you deliver harder working, better performing marketing campaigns, supply chain improvements and store operations execution. With capabilities like predictive customer analytics and others, you can identify and target the right audiences without going to a data analyst. The cognitive power helps you understand individual customer behaviors so you can create the ideal combination of interactions that drive conversion and build loyalty.
Ralph, thanks for expanding on the possibilities raised in the article. Jonathan Adler also has a great companion piece on Medium that talks about how to apply advanced techniques in a way that the business will actually use them. We still find that the vast majority of retailers are using only a fraction of the insights and analysis methods available today.
Most marketers we talk with have already adopted the approaches advocated in the article. Very few people would rely on a standalone number without applying context from the business.
I find the key elements of measurement plans that are core to successful adoption are to make sure the selection of measures are optimal to support objectives, accuracy is ensured and the output can be presented to clearly communicate a story about the business to stakeholders.
Checking off these items builds confidence in the measurement plan and facilitates decision making. Data analytics has always been about interpreting the numbers to create valuable insights that can be implemented for results.
I have found that in many cases, marketers (brand, agency and in-store) are reluctant to measure their initiatives because they are frankly afraid of what the results will show. Many seemingly clever, creative and expensive initiatives simply don’t work. In order to maintain funding or jobs it’s in their interest to keep the mystique.
If they do measure, marketers and merchants don’t take the time (or money) to understand the results, learn from the insights and then iterate and optimize the experience for both the shopper’s and vendor’s benefit. They simply create a large stack of graphs and charts that they can show their executives and claim that they are measuring for success. The data and its analysis itself is meaningless unless you’re prepared to listen, understand the insights and then have the courage to act upon them.
While KPIs are clearly important for helping us understand company performance, the challenge is in the old adage “That which gets metric-ed, gets managed.” Thus if the goal is to lift impressions or some other metric then the business shifts its focus to impacting this variable, sometimes at the expense of others, creating blind-spots for the organization. The point being, it’s important to look beyond with some exploratory research to see how the market, the consumer or the competitors are changing. This may necessitate a shift in both the tactics employed and the corresponding KPIs. Plus, as noted by others, context is everything and so keeping a lens towards the factors and situations impacting KPIs remains important to helping determine when it’s time to abandon the ship of an existing tactic or metric.
KPIs are important tools because they are simple to understand and can be easily compared between areas, managers and stores. Deeper analytics and predictive models help us understand what really happens and what we need to improve so our future results will be better.
Using advanced Big Data technology and AI marketers, retailers can get actionable insights and significantly improve their business results.
Nothing wrong with KPIs … you need them! What gets me is when the KPIs are assumed to be valuable, but their value has not actually been proven. To be valuable, a KPI must reflect the business growth model, have an established statistical relationship with performance at time t+1, align to a marketing action (if it goes down, here is what we do…) and collectively encompass the full range of marketing actions and business outcomes (including upstream behavioral markers, such as “test drives”). Brand equity statistics are the worst offenders … often no proven relationship to outcome (having been based on an assumption) and also never moving … that’s what makes them irrelevant as a brand tracker statistic.
I don’t think most marketers are overly reliant on KPIs or even data in general. Most of us now have more data than we need and the issue is around prioritization, assessment and then action based on the data. I do believe that there is a large void between people who can look and report on data and snapshots and those who can assess it, put it in context and most importantly make a business decision (hopefully a positive one) based on the data and their experience.
Leveraging tools that help make predictions based on data is amazing and can often help justify why a decision must be made but they do not help make the decision and then put the strategies, goals and tactics in place to put a decision into action.