Taking the Mystery Out of Mystery Shopping
By George Anderson
As with most market research, the information received from mystery shopper trips doesn’t do any good if a retailer doesn’t know what to do with it.
Jayne Keedy, president of JKS Inc. Market Research and Robert Smith, senior program manager for Maritz Research, spoke to NPN MarketPulse to offer suggestions on how store operators can gain insights that take the mystery out of why their stores are performing as they are.
Ms. Keedy said the value of mystery shopper programs is that they can help “you learn how most of your customers learn about your business.”
While data collected from mystery shoppers can’t answer all questions about a retail business, shopping research can be designed to help answer questions about customer service, a store’s appearance, product placement and loss prevention issues, she said.
Mr. Smith said mystery shopping is most valuable in assessing what he calls “curb appeal” — whether stores and bathrooms are clean; if store personnel are friendly and helpful. “Those are the things that will keep customers coming back,” he said.
Moderator’s Comment: How do mystery shopper programs fit in with other forms of research conducted by retailers to assess their business opportunities
and challenges? –
George Anderson – Moderator
I’ve always considered Mystery Shopper results as just one step above Mother-In-Law Research. It’s a snapshot of store conditions at a certain time of day, plus some anecdotal observations of store personnel. Neither has reliable value for helping management make improvements. District and regional management personnel should be observing store conditions on a regular enough basis to guide improvements, and anecdotal service observations can be obtained from customers in sufficient numbers to be statistically reliable.
My team once had the interesting task of mining mountains of Mystery Shopper reports for Kmart in their corporate HQ in Troy, MI, in a quest for a “silver bullet” to get the chain back on its feet. Everything, and I mean everything we learned had already been intuited by top management and given to us in separate briefings.
I know retailers who factor in mystery shopper info into store manager bonuses, which would certainly help. And I know store managers who actually pretend there are mystery shoppers to try to get their workers to do a better job with customer service. I think if the mystery shopper is well trained in merchandising, they can provide input on merchandising, but I’d mostly have them checking empirical things like in-stock conditions and such rather than passing judgment on matters of opinion. Of course, the ultimate mystery shopper is the senior HQ exec or CEO. That doesn’t happen nearly often enough.
One of the mysteries of poor shopping and service conditions at many retail outlets is how so much data can be collected, and yet the end result is still so bad. For example, related to mystery shoppers are shopper surveys to be filled out and returned for evaluation. Since many of them are filled out to complain about poor service, why do so many retailers have the surveys returned directly to the store, via the service desk? Do we assume that they will somehow, someway, find their way to appropriate managers at HQ? Or, don’t we really want to hear what’s on the cards?
No doubt a retailer can pick up an insight or two from mystery shopper reports. (Do you really need a mystery shopper to tell you if your bathrooms are clean?) But in terms of ‘research,’ a mystery shopper is like having a ringer in a game of pool or a pre-trained mouse in a maze experiment. The key is to measure the customer experience against customer expectations. A phony shopper ‘shops’ with phony expectations. The “how did we do?” kind of research in the absence of “what did you want?” has serious shortcomings.
Every non-store employee that visits a store should be required to complete a quick survey of condition. These are mailed to a central processing area for tabulation. This information should be matched with that of the mystery shopper. The real value of mystery shoppers is to develop a trend line. It is more important to see improvement or to identify problems/issues developing so corrective action can be taken.
I think too many retailers use mystery shoppers to drive certain behaviors that good leadership and good management should be doing instead. We’ve been moving our clients more towards customer experience measurement systems that are directly related to your experience strategy. Using a good IVR system with well defined questions to measure your customer’s response to your targeted in-store experience is much more important to me that paying someone to come into your store. What I also like is that every person walking in the door is a “mystery shopper” rather than the staff guessing who the one person a month is. The challenge of course is motivating your customer to give you the feedback in a timely manner. I also believe that all employees, and not just the store managers, should be compensated on the results. We also have clients who are using both shoppers and customer experience measurement systems together and are pleased with the balance approach.
I disagree with the majority of people who indicate that Mystery Shopping is a good tool for evaluating customer service.
A “mystery shopper” is a trained professional who is paid to anonymously visit/call your business and evaluate their experience based on a set of guidelines that the “shopper” was educated to observe. Mystery shopping is an effective tool for measuring operational standards. Mystery shopping is also useful to see if all locations are displaying promotional materials and offering specials in accordance with corporate guidelines. However, if your operations manual says all customers need to be offered a drink within the first five minutes of being seated, but your customers want the offer in three minutes, mystery shopping only confirms that you are consistently dissatisfying your customers.
Knowing how well your locations conform to your operations manual tells you little about how satisfied customers are with those standards or how customers’ needs may be changing over time. The experience of a “mystery shopper” and a customer is fundamentally different. “Mystery shoppers,” by necessity, receive training on how your business is supposed to be run. Because of this, a mystery shopper walks into your business knowing what is supposed to happen. They also know a great deal about the types of products and services you offer. Customers must try to learn this on their own through their experience with your business. Another important aspect to consider is that the mind-set of someone who is being paid to have an experience is fundamentally different from the mindset of someone who is paying to have an experience.
Your business’s idea of customer satisfaction is not always congruent with your customers’ idea of satisfaction. In addition, mystery shopping gives you no ability to identify how your customer expectations may be changing over time. A perfectly good operational plan one year may be hopelessly out of date the next.
Although a mystery-shopping program is a powerful tool for evaluating how your company meets operational standards, it is not an effective proxy for determining whether or not your customers are satisfied.
Al makes a great point. So many retailers do shops because that’s what retailers do rather than use them in a useful way. I’m always disappointed to see retailers use them as a device to “punish” employees rather using them as both a coaching tool and to reward the employees.
I’d like to amplify Tom’s comments above with some of my own.
Mystery shoppers cannot measure customer satisfaction. But they can observe whether certain retail practices are being executed according to “the book” – a narrow aspect of quality. Mystery shoppers are also sometimes employed to gather competitive price information.
Third parties that gather so-called “causal data” (Was my promotion implemented in a given store? Was my product in stock?) are doing follow-up work that I would classify separately from mystery shopping.
Retail service quality is a matter of consumer perception, and so it should be measured by periodically surveying consumers. There are well-established techniques for doing this (ServQUAL and similar scales). Unfortunately, retailers seem reluctant to make this investment, so they blunder on in the dark.
One must ask what information is being gained by the mystery shopper, what is the total cost of a corporate investment in the program, and what is the return and how will it be measured.
A large self-service chain with layers of poorly trained associates was once using mystery shoppers, thinking that it was consumer research. It just added to their costs, spied on their non-performing bureaucracy and gave their management symptoms of the disease but no cure. If management can’t monitor and spot consumer centric operating problems in their stores through their normal procedures, then the existing processes need to be changed or the chain’s management span of control needs to be re-evaluated.
Consumer and customer research are important strategic tools for setting a store’s merchandising and operating plans. Mystery shopping is a cover up for weak training and supervision; a management crutch for a chain with a broken limb.
As a mystery shopper provider, I essentially agree with everything Tom has to say. The simple fact of the matter is that mystery shopping measures store associate performance vis-Ã -vis corporate policy and standards. At its most basic application, mystery shopping is about compliance and it is to that extent that its impact is felt on the customer experience. How a store associate performs (by absolute predetermined corporate measures), does have an impact on the customer experience.
If one wanted to be truly honest, when mystery shopping is used as a compliance measure, the results should be read as indication of how well Human Resources and Training has done their job. Who is so brave to take that deep a look?
The real issue is that most companies use mystery shopping in the wrong way. Those companies that realize that mystery shopping can be a powerful tool for behavior modification find at the end of the day continued justification to support and fund the program and a proven ROI.
The concept of the mystery shopper is great to my way of thinking. Rather than interviewing hundreds of customers who may tell you things they want you to hear, the mystery shopper can report a very one off, personal impression of shopping the target store. Alternately, the president of the company may consider going on the sales floor and talking to customer one-on-one.
Personally, I offer my clients a mystery shopper program on steroids. It’s what I call the Clear Vision Audit. Leveraging over 30 years of retail development work, I walk through stores to experience the store as the customer would, virtually shopping every department in the store and interacting with employees. This is particularly useful with retailers who have a clearly defined marketing strategy, which is then tested to see if the promised service and product availability is actually delivered. Few achieve those marks.
So, my question is, besides looking at sales, or doing a secret shopper survey, what is the best way for a retailer to evaluate the customer’s experience?