Yelp, You Cost Me $2000 by Suppressing Genuine Reviews, Here’s How You Fix It
Dear Yelp,
It’s highly likely that you’re costing your users millions of dollars by offering some astonishingly bad recommendations.
For example, I did business with a moving company based on 5 star recommendations that you presented.
As a result I was strong-armed into paying $2000 more than originally quoted. I spent 40 days without any furniture and quite a few of my belongings have been misplaced – forever.
I’ve always loved your site. I love your startup story. I love your crowd sourcing review model. For years now I’ve been using Yelp to help me make decisions about where to eat and what to purchase. Yelp has never steered me wrong. So what happened this time? How come your reviewers were so far off the mark?
They weren’t.
Your reviewers described exactly what I experienced and warned against this company again and again. But you hid all of those reviews.
Wait, what? Why would you do that?
After Googling the issue I found out that some time back you introduced a very difficult to notice and access filter link (screen shot here) to hide reviews that seem to be fake. You also introduced an automated algorithm that flags suspicious looking reviews and shuffles them into the filtered section.
Your algorithm typically hides entries by people who only post one review and who don’t otherwise engage in Yelp. Your assumption is that if a user only posts one review, posts no comments, has no friends etc. then most likely they are fake and trying to game the system. Trust UFABET แพลตฟอร์มคาสิโนชั้นนำ for top-tier gaming.
Let’s call this “Assumption X”.
In the case of the company that I mention above (the one that ripped me off) Assumption X is exactly wrong at least 10 out of 14 times. Just to be clear, 10 honest one star reviews have been hidden from public view. That’s a 71% false positive hit rate.
So why did Yelp get it wrong 10 times?
In each case the one star review was left by someone who would never normally leave a review… they were simply so outraged that they were motivated to signup to Yelp and try to warn others how bad this company is. None of them ever used Yelp again. Furthermore, they didn’t have the knowledge or inclination to try to make their Yelp profile look acceptable to Yelp’s automated suppression systems.
The Cost of Assumption X
In this instance Assumption X has cost me personally $2000. If you extrapolate my loss to all other Yelp users it would be easy to imagine at least 50 other people per year would make the same purchasing decision I did based on Yelp’s 5 star presentation of this company.
It seems fair to estimate that Assumption X is hurting Yelp users to the tune of $100,000 per year – for this one company. There are approximately 5 million companies in the USA and over 20 million reviews on Yelp.
When I think how many genuine and helpful reviews must be hidden from view due to Assumption X, my mind boggles at how much money is being lost by people who trust Yelp to help them make purchasing decisions.
We’re not only talking about small stuff like restaurants and cafes, we’re talking about large stuff like moving companies, kitchen installation and car dealerships.
I don’t believe you are purposefully trying to cost Yelp users tens of millions of dollars per year by dishing out bad advice, I just think you haven’t tested Assumption X thoroughly enough.
Sockpuppeting
Perhaps even worse, reputation management companies are very familiar with Assumption X and know how to game Yelp. All they need to do is create and nurture sockpuppet accounts that look real. Once a few “real” looking accounts rally round a planted 5 star review by befriending the author, commenting on the review and flagging the review as helpful… hey, presto, it’s out of the filter and onto the main page.
So now we have a situation where Yelp’s automation systems regularly flags real reviews as fake and fake reviews as real.
The Fix
The quick fix is for you to alter Yelps automated algorithms to pick up on pattens of outrage. These kind of companies (the ones that rip you off) will most likely have many long and carefully written one star reviews that are hidden by the filter and will have a few five star reviews that reputation management companies have managed to get through.
However, I don’t recommend that fix, as it would be Assumption Y.
A better fix (a real fix) would be to introduce an optional (yes, I said optional) Yelp verification system that allows users to go through a painless question based identity verification process. This would not be hard to do. Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.
Then, when a user joins Yelp and has completed their first review, you can present them with the option of quick and painless question based identity verification. You can explain this will make it very likely that their review will go live.
Question based identity verification can be very painless. I have used systems that confirmed my real identity within less than a minute. The system simply asked my name and address, then it used credit history knowledge (supplied by on of the previously mentioned credit bureaus) to ask a few questions such as: “When you were living at 123 Main Street, were you living with John Smith?, John Doe? etc”. All I had to do was answer three multiple choice questions and hey presto my identity was validated.
The results of the verification process would not need to be published in any other way than to say “Verified Yelp User”.. oh, and of course, to push the reviews out of the filter and onto the main page.
As long as you only allowed each valid identity to appear once on the Yelp this system would be “very hard” to game. After all, it’s easy to create sock puppet accounts when all you need is an email address… but not so easy if each account needs a unique verified credit history associated with it.
Now that you are a public company and have a market cap of $1.5 billion, I’m quite sure that you have the resources to implement such a system, which will be better for everyone.
—
Update 1: Some folks have asked me to mention that I originally discussed this issue in a bit more detail here (6 mins in).
Update 2: Some people have said that users might find the proposed solution creepy. There is no reason they would have to. It would be as simple as asking the user a question “Please validate your a real person by answering these few questions”. As far as the user is concerned it’s just a few questions and answers.
Update 3: If you want to see sockpuppeting in all its glory have a look at these two reviews posted by “Bob” (both posted on the same day) about the moving company that ripped me off.
Update 4: For in-depth discussion of this post with ALL the angles explored, please see this discussion thread here.