Do customer reviews suffer from a herd mentality?
The belief that online user ratings are good indicators of product quality is largely an illusion, according to a new University of Colorado Boulder study.
Researchers examined user ratings for 1,272 products across 120 product categories, such as car seats, bike helmets, sunblock, air filters, smoke alarms and blood pressure monitors. Their analyses showed a very low correspondence between average user ratings of products on Amazon.com and product ratings, based on objective tests, found in consumer reports.
“The likelihood that an item with a higher user rating performs objectively better than an item with a lower user rating is only 57 percent,” said Bart de Langhe, author of the study and professor of marketing at CU-Boulder’s Leeds School of Business, in a statement. “A correspondence of 50 percent would be random, so user ratings provide very little insight about objective product performance.”
The researchers didn’t investigate the reasons reviews may be a poor indicator of product quality. However, they did point out that when reviewers comparing two products with the same objective qualities, they tend to give higher ratings to the one that is more expensive or from a brand with a premium reputation.
An often-heard criticism of online reviews is that a “herd mentality” leads those who read positive reviews to be more likely to leave positive remarks themselves. Social media also feeds this herding effect. Wrote Sinan Aral, the David Austin Professor of Management at the MIT Sloan School of Management, in a white paper, “We tend to herd on positive opinions and remain skeptical of the negative ones.”
- Consumers’ trust in online user ratings misplaced, says CU-Boulder study – University of Colorado Boulder
- Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings – University of Colorado Boulder
- The Problem With Online Ratings – MIT Sloan Management Review
- Can You Trust Online Reviews? – Kiplinger
Discussion Questions
DISCUSSION QUESTIONS: Should retailers attempt to do anything about bias that may creep into online customer reviews? Does it make sense to you that online customer reviews would tend to skew on the positive side?
Poll
BrainTrust
Richard J. George, Ph.D.
Professor of Food Marketing, Haub School of Business, Saint Joseph's University
Bill Hanifin
CEO, Hanifin Loyalty LLC
Retailers mess with consumer reviews at their own peril. Like them or not, consumer reviews are valuable to shoppers. And who’s to say that the CU Boulder study is accurate?
The positive skew is not surprising, particularly if a customer is keying into a particular brand for whatever reason, e.g., price, brand name, etc. The key is for retailers to hear what they don’t want to hear, namely the more negative comments. While the extremely positive comments (5s) are good for marketing purposes, they do little to help the retailer improve.
I suggest retailers throw out the outliers (1s and 5s) and concentrate on the 2-4 ratings. This is where the best learning will surface.
The tribe of digitally connected consumers regularly uses reviews as part of their decision-making process for purchase. I hazard the guess that most members of this tribe consume reviews with a “buyer beware” attitude.
I endorse light curation of reviews by retailers in order to monitor extreme opinions that are not helpful to any consumer, or those that are identified as having been planted with a purpose.
Beyond that, let reviews stand as they are. The online experience has matured to the point where consumers are capable of exercising discernment in their interpretation of the message.
Positive bias is mentioned in the article as skewing reviews, but negative bias (the mind’s predisposition to remember negative news) is equally powerful. Sure, there’s an element of self-fulfilling prophecy in online reviews but, when it comes down to it, the consumer knows whether they’re using a product they’d buy again. I’m always cautious of research which attempts to undermine the veracity of individual voices.
The construct of “star” reviews biases meaningful opinions to the two ends of the bell curve and getting things generally right, or as advertised, will win positive rating. My sense is that consumers are becoming skilled at reading into reviews as a composite scoring mechanism. I also think that growing consumer skepticism causes a perception bias against accepting positive reviews when there are negative reviews, on the basis that the most positive are posted by brands or their PR firms.
The age of the customer is clearly upon us. It’s certain that user reviews are an integral part of the shopping journey, and they are here to stay. With that said, retailers and all companies should steer clear of tampering with user reviews and retain the integrity of the system that consumers trust and depend upon.
While positive reviews are always welcome, retail organizations should focus on the exceptions, those 1 or 2 star, or even 5 star reviews, to better understand how poorly or how exceptional they are performing. As merchants and planners within a retail organization, companies should take an entrepreneurial stance with user reviews and leverage this as an opportunity to better engage and communicate with their customers.
I don’t see social media being high on positive comments as much as I see customers venting their frustrations. Sites like ConsumerAffairs.com, pissedconsumer.com, ripoffreport.com and many others are a voice for customers to share their true feelings about the product and the customer service. I think retailers need to look at social media and take what customers are saying seriously. Retailers lose business often because of poor service, and when you read the comments on these sites it makes you realize why the customer is unhappy and doesn’t recommend the product or the company. I share these sites with my clients and have found them to be very useful in convincing the retailer where they need to make changes. Looking at employees’ reviews also helps because if the employees are unhappy, that is going to affect customer service.
The process of consumer reviews is inherently biased, subjective and easily manipulated — positively or negatively. Reviewers take the time to review because they have some vested — economic, emotional, ideological or whatever — interest in making a statement about a product or service. And when you balance out the most likely motivations they tend toward the “positive” review side of the ledger. As for what retailers should do about negative reviews, I say nothing unless they are libelous or contain gross inaccuracies. Censorship is never a good way to make your case.
It makes a lot of sense that reviews skew to the positive. In my marketing career I have several times run projects asking consumers about future-planned products, with quite a few reacting that they already own the products and love them. They are trying to be helpful for what they think the researchers are looking for.
Six or eight months ago we needed to purchase a new refrigerator. Not having to do this for many years we did not know where to start. So we went window shopping and listened to a knowledgeable clerk. We were lucky because he knew what he was talking about and was not trying to sell us. We went home, went online and began our research which brought us back to the brand and model we saw in the store. Those reviews worked well for us.
Reviews should be important to the retailer or brand. Same for restaurants. They should want and need to know what their customers are saying when they don’t say it directly to them. I, for one, will tell a store manager if the food and service was good or bad in his restaurant before I leave.
An individual’s experience with a product is far more important than objective reviews. The former is personal and emotional, the latter is more logical and rational.
I would not mix the two nor would I compare them. Objective product reviews represent one set of decision criteria while customer reviews are another category all together.
Retailers should keep their hands out of anything that smacks of manipulating customer reviews.
“Do customer reviews suffer from a herd mentality?” HUMANS in general suffer from a herd mentality (but that’s probably a political topic). If you think about both retail and CPG brands, you can definitely draw conclusions that the most ubiquitous and loved brands inherently have some positive bias built into any potential reviews. The same can be said about brands with low value and sentiment. I think consumer reviews need to be taken in context with other shopper data points to paint a more scientific view of performance.
Whoever claimed that consumer reviews mirror product quality? They are more likely to reflect usability. Removing outliers is dangerous for two reasons. One is that many consumers like to see some outliers as an indication of credibility and as an indication of the kinds of things that upset consumers. The other is that if consumers find out that reviews are being curated, they are less likely to believe them and will give them the credibility they give to advertising.
It seems to me that review mechanisms are getting more sophisticated precisely to minimize the impact of some of these biases. I mean, they looked at whether better reviews resulted in more sales and said it barely contributed, but ask any marketplace seller on Amazon and they’ll tell you the FIRST review of a product — no matter what it is — can increase product sales by as much as 60 percent. So reviews still matter, a lot. But “guiding” consumer opinions by structuring what you ask their opinion on, for example by asking consumers to select from a range of attributes they feel apply to the product (“too big/too small” or “durable,” for example) — I think that’s totally okay. Paying for reviews, whether with points, product or discounts, is not. I see plenty of people who are sophisticated about how to absorb reviews, too, and who are at least somewhat aware of their own biases when they read reviews. So I’m not sure it’s as bad as the research makes out.
Relying upon user ratings to measure a product’s success is unreliable because the data is limited to 1.) the kind of person that would take the time to provide a rating (as opposed to all users); 2.) the mood of the users who provide the rating at the time they are giving the rating (did they just have a bad experience with the product?); and 3.) the differences in how the people willing to provide ratings perceive the given ratings measurements — good to me might be excellent to someone else.
In a study we did, online reviews rated as more important to purchase intent than touch and feel, especially with young people. So the last thing you want to do in my opinion is tamper with said reviews. That would be damaging the brand.
The issue we’re discussing falls back on the openness of the internet itself and how retailers who were raised in an earlier era have problems with the lack of control. In this instance, just like social media (think: Glassdoor), you’ve got to focus on quality across the board and let the chips fall where they may. As they say in sports: focus on what we do, it’s the only thing we can control.
Customer review ratings depend on many factors: complete user experience with an item, in-store customer service, person’s emotional state, return policy and many others. These factors can influence one’s ability to think objectively. I have to disagree that customer reviews are skewing toward the positive side due to the “loss aversion principal.” This states that people tend to prefer avoiding losses to acquiring equivalent gains. That is to say, losses are twice as powerful psychologically. People are more likely to leave a bad comment than a good one. According to Dr. Kohei Kawamura, people find all-positive (five-stars) reviews less credible if there are no bad ones presented.
To make reviews more objective, customers should be presented with simple yes or no answer options as opposed to the 5-stars review system. In short, retailers should focus on providing the best possible customer experience and deal with negative reviews fast.
There is a psychological point behind this. A good consumer review may beget more of the same. If my friends tell me the restaurant I’m going to tonight is going to be great, I’ll believe. The success of my meal has started before I even get there. I’ll be looking for ways to confirm my friends’ positive comments. The opposite is also true. Tell me it’s lousy and I’ll find fault much quicker.
Still, I am a firm believer that the online review is important and powerful. It not only shows how customers/consumers think about the product, it also shows how a company reacts to the reviews and engages with their customers.
Customer reviews are off limits for retailers. There is the herd brain set. But that is the choice of the herds. Herds are more prominent in major network news. Most people watching it flow like a herd and believe all of it — without learning more on their own.
Before I make a decision on big ticket stuff I look at reviews, articles, and specs.
Same with news — before I believe it I research it and make sure it what I heard or read is not the canned content of a network that controls or tries to control the herd.
Example: the recent NBC, CBS, etc. “end of retail stores” news push. The industry went up 4.6% yet press spins the end of retail as we know it. Stop!!!
To balance things out and to increase credibility, retailers should supply consumers with third party evaluations of their products, in addition to consumers reviews.
It is not necessarily a herd mentality. There are types of consumers that are likely to leave reviews (such as young people, people that have had terrible experiences or extremely good experiences). Increasingly, the reminders that retailers and sellers send out to buyers provide incentives for more and more consumers to leave reviews that are actually reflective of the experience they had.