Researchers Sniff Out Fake Reviews Online

Online reviews are under scrutiny again by the folks at Cornell. Last month, RetailWire ran a story on research conducted by Trevor Pinch, a professor at the school, who surveyed top reviewers at Amazon and discovered they tended to offer a higher percentage of positive reviews than others lower on the site’s ranking list.

Now, new research shows that opinion spam — phony reviews to prop up or disparage a product — are hard to detect by humans looking to edit them out. Software used by Cornell’s researchers, however, was able to track fake reviews with close to 90 percent accuracy, according to the Cornell Chronicle.

Myle Ott, a Cornell graduate student involved in the research, said that human editors have a “truth bias.” They begin by assuming reviews are honest unless shown otherwise. On the other hand, once their bias has been shaken, they tend to go too far in the other direction assuming many reviews are fake when they are not.

A computer analysis of false and legitimate reviews found some key distinctions between the two. Fake reviewers tend to use more verbs and fewer nouns than those on the up-and-up.

A review of hotel reviews found that honest reviewers were more concrete in their language with specific references to terms such as “bathroom” or “price.” Those looking to skew results used terms such as “business trip” or “vacation.”

Mr. Ott said the software developed by Cornell could be used as “first-round filter” for sites that have high levels of fake reviews.

“Ultimately, cutting down on deception helps everyone,” he told the Chronicle. “Customers need to be able to trust the reviews they read, and sellers need feedback on how best to improve their services.”

BrainTrust

Discussion Questions

Discussion Questions: How big of a problem do you think “opinion spam” is for ecommerce sites? What can retailers, brands, etc. do to make the system more honest?

Poll

12 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Joan Treistman
Joan Treistman
12 years ago

Another way of stating the tendency of humans to consider reviews honest is that “we see things the way we want to see them.” Of course I think it’s just a matter of time before there is software to create spam reviews…which will contain just the right amount of nouns and verbs to fool the Cornell software.

Here’s my way of injecting a truth serum for vacation reviews. I look for the British and German reviewers. They are more forgiving of certain aspects of a hotel or resort, but demand certain basics I agree with.

For restaurant reviews I examine the details of why people liked the restaurant and why others disliked the restaurant. I align the attributes along my own importance scale and make a decision. When there are no specifics regarding the food and service, I pass.

Bernice Hurst
Bernice Hurst
12 years ago

It would be useful to have a handy list of key phrases. I don’t think many people consider whether reviews are fake unless and until they have a totally different experience. And then there are people who like me who are suspicious of all….

David Biernbaum
David Biernbaum
12 years ago

Fake reviews for CPG are definitely an issue because an increasing number of consumers are heavily influenced by online reviews, in particular, those posted by other “consumers.” The more qualifying a review, the better. If it were my business, I would not allow reviews from consumers that are not able to prove that they purchased the product, and are indeed, “consumers.”

Anne Howe
Anne Howe
12 years ago

I think most consumers read reviews, and like Joan said, line up the review comments with their own personal set of criteria. It’s hard to overcome personal bias when looking over reviews since after all, you’re most likely looking at them to satisfy a desire to have something, be it a product or an experience. So a plethora or good or bad comments may tip the scale to a decision.

To any online business, getting a reputation for posting fake reviews could be extremely damaging to brand perception. If good software is available, it’s time to invest in it to vet reviews. And to post that reviews have been “scanned for spam.” There’s too much on the line in this day and age to risk the loss of shopper trust.

James Tenser
James Tenser
12 years ago

Taken together, the present and prior stories point toward a rather troubling possibility for manipulating online opinion.

There is potential for certain online marketing entities to use “crowd-sourcing” techniques to pump out reviews that favor the interests of clients. So, if an adverse story hits the media, offer small bounties to individual “freelancers” who post favorable items on social sites, blogs and review sites.

The effect would be to overwhelm the bad press items and bury them at the bottom of search results. Or to influence twitter “trends” and other apparent indicators of public opinion. The same methods could apply to hype a new product introduction or penny stock, or to sabotage a competitor’s reputation.

Alas, the potential for such manipulation is the flip side of the “democratizing” promise of the World Wide Web. When anyone may post any opinion for any reason, freedom to tell the truth is upheld along with freedom to lie.

Which leads me to suspect that opinion spam is a sore temptation in the political realm.

Craig Sundstrom
Craig Sundstrom
12 years ago

I agree that honesty is a problem with amateur reviews, but it is just one of many. The bigger problem is the “amateur” part: people too harsh, too lenient, or in many cases just too stupid to write anything useful. Sure, in most cases if you actually read the reviews, you can separate the kernels from the chaff, but who wants to engage in a research project just to buy a sandwich? I think, in the long run, most will find this element of the Net’s “democracy” to be less-than-helpful. Let the masses eat their cake if they want, but they can keep their opinions to themselves.

Rick Moss
Rick Moss
12 years ago

I heard just the other day from a reliable source who has a high-performance auto parts business that these “opinion spammers” are easy to hire. He claims that others in his business use them to enter negative comments in car enthusiast forums in order to denigrate their competitors’ products. These spammers are knowledgeable about this specialty and “sniffing them out” is tough — at least for human beings.

The development of helpful automated programs to identify these bogus postings sounds promising. Unfortunately, the spammers are often smart enough to stay one step ahead of the spammer-trackers. My advice for those that allow consumer comments on your site — require registration so a site admin can at least check out if the poster looks legit. It’s by no means foolproof but at least adds a layer of nuisance for spammers and may be enough to dissuade them.

Bill Hanifin
Bill Hanifin
12 years ago

One way to more highly qualify reviews is to require that a purchase has been made in order to post. Zavee.com has this business rule in place as part of its social shopping network in South Florida.

I also wonder if there is a difference in “opinion spam” between positive and negative reviews. I would think that it is much easier to craft a fake positive review and that it happens more often than the reverse.

Consumers wishing to share a negative experience will take the time to do so and I’ll bet the percentage of authenticity across negative reviews is higher than the positive.

M. Jericho Banks PhD
M. Jericho Banks PhD
12 years ago

As a consumer, I respect professional reviewers but ignore ad hoc customer comments. For instance, I trust the folks at CNET and ZDNet, but not the volunteered reviews on their sites. Forums are different in that many of them provide solutions based on personal experience rather than post-purchase opinions for pre-purchase shoppers.

As an ecommerce business owner on the other hand, I pay attention to all reviews that provide specific areas of satisfaction or dissatisfaction (not just “it sucks”). Genuine or not, they sometimes point to concerns I need to address. Also, we’re lucky in that the very few negative comments about us which appear here and there are brought to our attention by potential customers asking for clarification. It’s the nature of our type of business, and our support staff is totally prepared to offer precise responses. In short, we do not perceive opinion spam to be a problem for us.

Doug Stephens
Doug Stephens
12 years ago

What is also interesting is that even REAL reviews are highly prone to skewing. The more reviews a particular product or topic have, the more prone reviewers are to giving unrealistically low or high reviews – just in an effort to get their review noticed. So, it’s probably a good idea to throw out the really low and high reviews and focus on those in the middle for the best sense of what a product or service is really like.

Ralph Jacobson
Ralph Jacobson
12 years ago

Cool technologies like this can be and are being used by companies globally. It can help make the shopping experience far more credible and it will also drive loyalty to the site if the shopper has a positive experience.

Andrew Pettit
Andrew Pettit
12 years ago

The ability to verify whether or not a reviewer is being honest about who they are is definitely lacking in the current online marketplace. Reading reviews is still a critical element in many online shoppers’ research process, and thus integral to the ecommerce experience, but there are more social solutions.

The growth of on-site Social Q&A in the social commerce sector helps to provide this degree of verification. In order to answer a shopper’s question via Social Q&A, the responder must be a past customer. The shopper immediately knows that the source from which they are deriving their information is a verified customer who purchased that product.

With more and more sites focusing on social commerce strategies, I wouldn’t be surprised to see more online retailers implementing on-site Social Q&A to improve the quality of their customers’ research experience. Don’t get me wrong, conventional reviews do serve as the first step in a shopper’s research process, and that will continue. However, as shoppers become more savvy about the inability to properly verify reviews, they’ll want more tools like Social Q&A to improve their experience.