I’ve always considered Mystery Shopper results as just one step above Mother-In-Law Research. It’s a snapshot of store conditions at a certain time of day, plus some anecdotal observations of store personnel. Neither has reliable value for helping management make improvements. District and regional management personnel should be observing store conditions on a regular enough basis to guide improvements, and anecdotal service observations can be obtained from customers in sufficient numbers to be statistically reliable.

My team once had the interesting task of mining mountains of Mystery Shopper reports for Kmart in their corporate HQ in Troy, MI, in a quest for a “silver bullet” to get the chain back on its feet. Everything, and I mean everything we learned had already been intuited by top management and given to us in separate briefings.

The concept of the mystery shopper is great to my way of thinking. Rather than interviewing hundreds of customers who may tell you things they want you to hear, the mystery shopper can report a very one off, personal impression of shopping the target store. Alternately, the president of the company may consider going on the sales floor and talking to customer one-on-one.

Personally, I offer my clients a mystery shopper program on steroids. It’s what I call the Clear Vision Audit. Leveraging over 30 years of retail development work, I walk through stores to experience the store as the customer would, virtually shopping every department in the store and interacting with employees. This is particularly useful with retailers who have a clearly defined marketing strategy, which is then tested to see if the promised service and product availability is actually delivered. Few achieve those marks.

As a mystery shopper provider, I essentially agree with everything Tom has to say. The simple fact of the matter is that mystery shopping measures store associate performance vis-à-vis corporate policy and standards. At its most basic application, mystery shopping is about compliance and it is to that extent that its impact is felt on the customer experience. How a store associate performs (by absolute predetermined corporate measures), does have an impact on the customer experience.

If one wanted to be truly honest, when mystery shopping is used as a compliance measure, the results should be read as indication of how well Human Resources and Training has done their job. Who is so brave to take that deep a look?

The real issue is that most companies use mystery shopping in the wrong way. Those companies that realize that mystery shopping can be a powerful tool for behavior modification find at the end of the day continued justification to support and fund the program and a proven ROI.

One must ask what information is being gained by the mystery shopper, what is the total cost of a corporate investment in the program, and what is the return and how will it be measured.

A large self-service chain with layers of poorly trained associates was once using mystery shoppers, thinking that it was consumer research. It just added to their costs, spied on their non-performing bureaucracy and gave their management symptoms of the disease but no cure. If management can’t monitor and spot consumer centric operating problems in their stores through their normal procedures, then the existing processes need to be changed or the chain’s management span of control needs to be re-evaluated.

Consumer and customer research are important strategic tools for setting a store’s merchandising and operating plans. Mystery shopping is a cover up for weak training and supervision; a management crutch for a chain with a broken limb.

I’d like to amplify Tom’s comments above with some of my own.

Mystery shoppers cannot measure customer satisfaction. But they can observe whether certain retail practices are being executed according to “the book” – a narrow aspect of quality. Mystery shoppers are also sometimes employed to gather competitive price information.

Third parties that gather so-called “causal data” (Was my promotion implemented in a given store? Was my product in stock?) are doing follow-up work that I would classify separately from mystery shopping.

Retail service quality is a matter of consumer perception, and so it should be measured by periodically surveying consumers. There are well-established techniques for doing this (ServQUAL and similar scales). Unfortunately, retailers seem reluctant to make this investment, so they blunder on in the dark.

Al makes a great point. So many retailers do shops because that’s what retailers do rather than use them in a useful way. I’m always disappointed to see retailers use them as a device to “punish” employees rather using them as both a coaching tool and to reward the employees.

I disagree with the majority of people who indicate that Mystery Shopping is a good tool for evaluating customer service. A “mystery shopper” is a trained professional who is paid to anonymously visit/call your business and evaluate their experience based on a set of guidelines that the “shopper” was educated to observe. Mystery shopping is an effective tool for measuring operational standards. Mystery shopping is also useful to see if all locations are displaying promotional materials and offering specials in accordance with corporate guidelines. However, if your operations manual says all customers need to be offered a drink within the first five minutes of being seated, but your customers want the offer in three minutes, mystery shopping only confirms that you are consistently dissatisfying your customers. Knowing how well your locations conform to your operations manual tells you little about how satisfied customers are with those standards or how customers’ needs may be changing over time. The experience of a “mystery shopper” and a customer is fundamentally different. “Mystery shoppers,” by necessity, receive training on… Read more »

I think too many retailers use mystery shoppers to drive certain behaviors that good leadership and good management should be doing instead. We’ve been moving our clients more towards customer experience measurement systems that are directly related to your experience strategy. Using a good IVR system with well defined questions to measure your customer’s response to your targeted in-store experience is much more important to me that paying someone to come into your store. What I also like is that every person walking in the door is a “mystery shopper” rather than the staff guessing who the one person a month is. The challenge of course is motivating your customer to give you the feedback in a timely manner. I also believe that all employees, and not just the store managers, should be compensated on the results. We also have clients who are using both shoppers and customer experience measurement systems together and are pleased with the balance approach.

Every non-store employee that visits a store should be required to complete a quick survey of condition. These are mailed to a central processing area for tabulation. This information should be matched with that of the mystery shopper. The real value of mystery shoppers is to develop a trend line. It is more important to see improvement or to identify problems/issues developing so corrective action can be taken.

No doubt a retailer can pick up an insight or two from mystery shopper reports. (Do you really need a mystery shopper to tell you if your bathrooms are clean?) But in terms of ‘research,’ a mystery shopper is like having a ringer in a game of pool or a pre-trained mouse in a maze experiment. The key is to measure the customer experience against customer expectations. A phony shopper ‘shops’ with phony expectations. The “how did we do?” kind of research in the absence of “what did you want?” has serious shortcomings.

One of the mysteries of poor shopping and service conditions at many retail outlets is how so much data can be collected, and yet the end result is still so bad. For example, related to mystery shoppers are shopper surveys to be filled out and returned for evaluation. Since many of them are filled out to complain about poor service, why do so many retailers have the surveys returned directly to the store, via the service desk? Do we assume that they will somehow, someway, find their way to appropriate managers at HQ? Or, don’t we really want to hear what’s on the cards?

I know retailers who factor in mystery shopper info into store manager bonuses, which would certainly help. And I know store managers who actually pretend there are mystery shoppers to try to get their workers to do a better job with customer service. I think if the mystery shopper is well trained in merchandising, they can provide input on merchandising, but I’d mostly have them checking empirical things like in-stock conditions and such rather than passing judgment on matters of opinion. Of course, the ultimate mystery shopper is the senior HQ exec or CEO. That doesn’t happen nearly often enough.

So, my question is, besides looking at sales, or doing a secret shopper survey, what is the best way for a retailer to evaluate the customer’s experience?