How can new product forecasting be improved?
According to a study from Martec International and Relex, the biggest challenge for North American retailers regarding forecasting is around new products.
A whopping 73 percent of North American retailers cited new product forecasting as a challenge, notably above the response from overall global retailers at 58 percent. The study was based on 126 interviews across North America (Canada and U.S.), Germany, U.K. and Nordic countries (Denmark, Finland, Norway and Sweden).
“The U.S. tends to be the country where most new products are developed and launched and so this causes greater problems for North American retailers than elsewhere,” Martec noted in its study.
Among the supply chain challenges involved with new product forecasting noted in the study for all retailers:
- Only 42 percent are able to show new product pipeline stock build in their supply chains;
- Only 39 percent are able to automatically allocate new products within desired business requirements;
- Only 29 percent can build new product demand forecast automatically at SKU/store level.
Other major forecasting challenges for the North American retailers surveyed include weather, cited by 64 percent, and promotions and seasonal items, both 52 percent.
With estimates of new product failures at between 50 percent and 80 percent, the challenges aren’t surprising.
New product forecasting is plagued by a lack of product history or an uncertain product life cycle. While products creating new categories are the hardest, gauging demand for extensions to existing products or refinements (i.e., “new and improved”) also face major hurdles.
Some sophisticated software that better clusters and reviews past new product introductions are promising to improve new product forecasting. Tapping social media’s reach and predictive analytics, enhanced consumer survey methods are likewise promising to better predict demand.
First Insight, for example, surveys consumers to predict which fashion styles will perform well for particular retailers. Founder Greg Petro recently told Women’s Wear Daily that new-product forecasting “requires educated risk-taking.”
- State of the Retail Supply Chain 2016 – Martec International
- Martec & Relex Solutions Present ‘State of the Retail Supply Chain 2016’ – Relex
- Demand forecasting for new products? Many heads are better than one! – TIAS
- First Insight’s Greg Petro Explains the Importance of Data – Women’s Wear Daily (sub. required)
Discussion Questions
DISCUSSION QUESTIONS: Do you see enhanced software or survey methods improving the ability to forecast new products? How much will retailers continue to rely on intuition and hunches when attempting to predict new product demand?
Poll
BrainTrust
Ryan Mathews
Founder, CEO, Black Monk Consulting
Anne Howe
Principal, Anne Howe Associates
I’m surprised nobody starts with the manufacturer’s estimate — are there any new CPG products introduced without manufacturer research? Surely they have a forecast which they can filter down to the chain level. I would have thought that would be the retailer’s starting point.
Look — this is a perennial problem with a simple solution — talk to the customer and watch how they shop in other areas.
The reason new product introduction is so problematic is that it is a “push” system in a “pull” world. We should be designing from real people back to manufacturing, not the other way around.
We should worry less about what we want to sell and more about what real people want to buy.
The fashion retailers have new product introduction processes built into their rollouts: introduce a small selection of items, keep real-time tabs on sales and replenish the stock daily based upon sending sales results/orders to the manufacturers set up to respond quickly. When there is no historical data, retailers can either rely solely on their forecasts (which are inevitably wrong) or build a system responsive to sales.
There has been a gradual but definite move away from seat-of-the-pants forecasting to science-based forecasting over the last 20 years. The timing of this trends correlates almost exactly to the retiring of the old merchandisers who kept stacks of computer read outs that showed transaction log details in the corners of their office and the entrance of managers with the latest analytical skill sets.
There are pros and cons to this shift away from the art of forecasting and to the science. On the pro side, there is standardization and easy measurement of success and failure. There’s also considerable automation that enables managers to spend less time on the regular price changes and promotions and concentrate on problems. On the con side, there is a considerable disconnect between merchandisers and shoppers that results in bigger misses.
There is no stopping the technology innovations being thrown at forecasting. I just hope vendors come up with algorithms that emulate the art part of forecasting.
This has got to be a tight collaboration between the retailer and the CPG supplier. The lack of collaboration has resulted in these poor RTE statistics for decades. Yes there are tools available today, and some great new ones on the horizon, that leverage the best cognitive machine learning capabilities that make forecasts better and better over time, based on all the information that is fed into the modelers. This information includes myriad external forces, like weather (already mentioned in the article), social chatter and influencers, news items, local events and many others. We have worked with retailers to see improvements of not only new item forecasts, but also in how existing items may or may not be impacted by these new items. The potential affinities or relationships between new and existing items can be forecast down to the SKU level now. While “gut feel” may still have a place to some degree, I am confident these new technologies will reduce the human error down to a minimum in the near future.
Could software and sophisticated forecasting algorithms improve new product forecasts? Perhaps. However, there are far too many “soft” factors to rely solely on the data. In fact, the advantage of applying human factors (both from consumer research around acceptance of the item into retail and from the category manager’s gut feel) is that intuition and educated guesses matter.
Because new items remain the lifeblood of category growth I am concerned that the empowerment once given to category managers who were making decisions as “merchants” has been replaced by forecasting models that are devoid of experience and hunches. Even in a perfect world, I believe there is no perfect world for new item forecasting.
At the risk of sounding like a broken record, I think better research into understanding unmet emotional needs can vastly improve new product innovation and forecasting. Are retailers averse to adding funds to what many manufacturers are doing in new product innovation? The distinct lack of real collaboration seems to continue on as standard practice and I, for one, don’t get it. Serve the shopper via deeper understanding and investment. What is so hard about that? Shopper-back or shopper-first, the real leaders get it and fund it.
This is, perhaps, the most difficult of problems. And it involves a set of very real tensions that aren’t easily solved.
Retailers need to control risk with new products — and tend to be conservative in order to do that. But by doing so, they often cut back orders to levels where manufacturers can’t afford to ramp up production at reasonable scale.
Further, new products may have an opportunity window that closes quickly. All this puts pressure on initial orders to be substantial enough to grasp the opportunity.
This fight between opportunity and risk is key. Retail teams need to stay away from the extremes. On the one hand, many exert so much caution that risk is increased. On the other hand, when these cautious teams do take risks, they can jump to the other extreme.
It’s my experience that each retailer needs a team skilled at new product work to beat not simply the inventory question, but the need for advertising/communication support, need for in-store execution to support new products, and developing a pipeline of strong new products to keep the stores fresh and interesting.
Enhanced software or survey methods won’t improve forecasting ability. Having worked a career in new products, they’re so new consumers can’t give accurate answers.
So the merchant teams need to rely on both data and intuition, AND on making choices informed by research and data. But most important, they need executive backing for the reality that risk is critical to retail success.
This is not an easy question, nor is there an easy solution. Many ways have been tried. Some work occasionally, others never make it past the first use. We have become so tech based and focused that this seems to be the only method to use. But how to develop the software to rely on is an unanswered and unsolved mystery. It all comes down to whether the public will grab the bait thrown out by marketing.
Example: I bought a newer, greater smartphone two months ago because it appeared to meet my needs at the time. It did — but now something that fits my needs better has reached the sales floor. Oh well, only thirty four months to go on this contract.
If the data used to generate reports in support of decision making is irrelevant or valueless then no amount of any kind of software is going to help. This answer to the question is self evident perhaps in appearance only. The true problem is not having the sales, marketing and IT people on board that can construct and maintain a decision management plan and capability for new business development. And before you put that together, it is necessary to know what you want to do and if it is possible. It is a shame that so many prominent business positions are filled with people that guess on their own or along with less talented subordinates. Is it any wonder that the depression seems endless?
Hear Hear, Ryan Mathews. Luckily, some retailers are just beginning to move beyond traditional new product market research, and are looking at previewing planned new products at several digital touchpoints by simply asking consumers about their interest in a future purchase. Viewers are offered the option to opt-in to triggered alerts on the path-to-product-launch, much like Kickstarter does. And a leading retail supply chain forecasting software vendor is partnering with this consumer-facing feedback system to help prepare the retailers and vendors for making smarter buying decisions. I’m also reminded of the businesses that monitor Amazon product comments to discover unmet consumer needs and then design and rush those goods into production.
To some extent, intuition and hunches will still play a role. Forecasting new products is both an art and a science — though the science part will become increasingly sophisticated.