Yelp, You Cost Me $2000 by Suppressing Genuine Reviews, Here’s How You Fix It

Dear Yelp,

It’s highly likely that you’re costing your users millions of dollars by offering some astonishingly bad recommendations.

For example, I did business with a moving company based on 5 star recommendations that you presented.

As a result I was strong-armed into paying $2000 more than originally quoted. I spent 40 days without any furniture and quite a few of my belongings have been misplaced – forever.

I’ve always loved your site. I love your startup story. I love your crowd sourcing review model. For years now I’ve been using Yelp to help me make decisions about where to eat and what to purchase. Yelp has never steered me wrong. So what happened this time? How come your reviewers were so far off the mark?

They weren’t.

Your reviewers described exactly what I experienced and warned against this company again and again. But you hid all of those reviews.

Wait, what? Why would you do that?

After Googling the issue I found out that some time back you introduced a very difficult to notice and access filter link (screen shot here) to hide reviews that seem to be fake. You also introduced an automated algorithm that flags suspicious looking reviews and shuffles them into the filtered section.

Your algorithm typically hides entries by people who only post one review and who don’t otherwise engage in Yelp. Your assumption is that if a user only posts one review, posts no comments, has no friends etc. then most likely they are fake and trying to game the system.

Let’s call this “Assumption X”.

In the case of the company that I mention above (the one that ripped me off) Assumption X is exactly wrong at least 10 out of 14 times. Just to be clear, 10 honest one star reviews have been hidden from public view. That’s a 71% false positive hit rate.

So why did Yelp get it wrong 10 times?

In each case the one star review was left by someone who would never normally leave a review… they were simply so outraged that they were motivated to signup to Yelp and try to warn others how bad this company is. None of them ever used Yelp again. Furthermore, they didn’t have the knowledge or inclination to try to make their Yelp profile look acceptable to Yelp’s automated suppression systems.

The Cost of Assumption X

In this instance Assumption X has cost me personally $2000. If you extrapolate my loss to all other Yelp users it would be easy to imagine at least 50 other people per year would make the same purchasing decision I did based on Yelp’s 5 star presentation of this company.

It seems fair to estimate that Assumption X is hurting Yelp users to the tune of $100,000 per year – for this one company. There are approximately 5 million companies in the USA and over 20 million reviews on Yelp.

When I think how many genuine and helpful reviews must be hidden from view due to Assumption X, my mind boggles at how much money is being lost by people who trust Yelp to help them make purchasing decisions.

We’re not only talking about small stuff like restaurants and cafes, we’re talking about large stuff like moving companies, kitchen installation and car dealerships.

I don’t believe you are purposefully trying to cost Yelp users tens of millions of dollars per year by dishing out bad advice, I just think you haven’t tested Assumption X thoroughly enough.

Sockpuppeting

Perhaps even worse, reputation management companies are very familiar with Assumption X and know how to game Yelp. All they need to do is create and nurture sockpuppet accounts that look real. Once a few “real” looking accounts rally round a planted 5 star review by befriending the author, commenting on the review and flagging the review as helpful… hey, presto, it’s out of the filter and onto the main page.

So now we have a situation where Yelp’s automation systems regularly flags real reviews as fake and fake reviews as real.

The Fix

The quick fix is for you to alter Yelps automated algorithms to pick up on pattens of outrage. These kind of companies (the ones that rip you off) will most likely have many long and carefully written one star reviews that are hidden by the filter and will have a few five star reviews that reputation management companies have managed to get through.

However, I don’t recommend that fix, as it would be Assumption Y.

A better fix (a real fix) would be to introduce an optional (yes, I said optional) Yelp verification system that allows users to go through a painless question based identity verification process. This would not be hard to do. Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.

Then, when a user joins Yelp and has completed their first review, you can present them with the option of quick and painless question based identity verification. You can explain this will make it very likely that their review will go live.

Question based identity verification can be very painless. I have used systems that confirmed my real identity within less than a minute. The system simply asked my name and address, then it used credit history knowledge (supplied by on of the previously mentioned credit bureaus) to ask a few questions such as:  “When you were living at 123 Main Street, were you living with John Smith?, John Doe? etc”. All I had to do was answer three multiple choice questions and hey presto my identity was validated.

The results of the verification process would not need to be published in any other way than to say “Verified Yelp User”.. oh, and of course, to push the reviews out of the filter and onto the main page.

As long as you only allowed each valid identity to appear once on the Yelp this system would be “very hard” to game. After all, it’s easy to create sock puppet accounts when all you need is an email address… but not so easy if each account needs a unique verified credit history associated with it.

Now that you are a public company and have a market cap of $1.5 billion, I’m quite sure that you have the resources to implement such a system, which will be better for everyone.

Update 1: Some folks have asked me to mention that I originally discussed this issue in a bit more detail here (6 mins in).

Update 2: Some people have said that users might find the proposed solution creepy. There is no reason they would have to. It would be as simple as asking the user a question “Please validate your a real person by answering these few questions”. As far as the user is concerned it’s just a few questions and answers.

Update 3: If you want to see sockpuppeting in all its glory have a look at these two reviews posted by “Bob” (both posted on the same day) about the moving company that ripped me off.

Update 4: For in-depth discussion of this post with ALL the angles explored, please see this discussion thread here.

The Only Reason My App Worked Was Due to a Slow Database

Things Are Not What They Seem

Astonishingly, the only reason my Ajax driven web app has been able to run consistently and successfully for the past two years is due to MySQL latency.

I know this because yesterday (with the help of Jonathon Hill) I optimized the database by moving it off the server and onto it’s own 4GB Rackspace instance. (Everything was running on a single 2GB Rackspace instance prior to this).

As soon as I got the new database up and running it was blazingly fast (using the Percona flavor of MySQL made it a breeze).

But today I noticed that the load average on the front end head server (now MySQLless) was running super high (100+) with customers being unable to login and generally use the service.

After some digging around I noticed that Ajax requests were being served wayyy faster allowing for each client to establish 5-10 concurrent ajax requests thus flooding the Apache child pool – amping up the load – and bringing down the site.

Semi-blocking Ajax Queue

To fix this I quickly implemented a semi-blocking queuing system for all Ajax requests.

Basically the JavaScript now queues all ajax requests the client sends and only makes one ajax call at a time UNLESS a call is blocked for more than one second, then it is allowed through.

This allows the javaScript to be asynchronous but severely throttles asynchronousness (if that is a word!) concurrency.

I was expecting this to slow things down but it seems to have really speeded things up from both a user and backend perspective.

But perhaps even more importantly the server load average has gone from 100+ down to 0.2.

I can only assume the reason the system has been working for the past two years is because MySQL latency was forcing the Apache child processes to run slower, with a knock on effect of Ajax responses being slower, with a knock on effect of chained Ajax requests being delayed thus creating a less asynchronous environment.

In other words, the only reason my app worked for the past two years was due to a slow database!

How I Converted My Subscription Site from Paypal to Stripe in 2 Days

PayPal to Stripe in 2 Days

Stripe.com is a dream for any developer wanting to build payment processing into their site.

In fact, the API is so easy to use it leaves me wondering why many other API’s make such a mountain out of a molehill. Stripe goes one step further by making every part of website payment integration stupidly easy.

App Store Approval = Days   –>   Stripe Approval = Instant

For example, have you ever tried to get approval to join the App Store and start making money? It takes hours of form filling and fact checking, then days of waiting. With stripe you only need to fill out a single form with your company information and tax ID. Then click a button and – bada-boom bada-bing – you can start collecting money online instantly.

(Assuming you live in the USA and have your company papers in order.)

PayPal Sandbox = Hours   –>   Stripe Sandbox = Instant

Another example is with PayPal, if you want to use the sandbox to test your application it takes hours of fiddling around setting up a sandbox account loging in and out of various vendor, seller accounts etc. But with stripe it is as simple as clicking a swich that says “test” and using a pre-assigned test API key.

Even better than that, all the documentation auto inserts a working test API key into any example code so you can literally copy/paste from their site into yours and things work.

Merchant Account = Headache   –>   Stripe = Merchant Account

You probably already know this, but just in case you didn’t, stripe does not require a merchant account. Simply enter your existing bank routing and account number and stripe sweeps money into your account on a daily basis (with a 7 day lag per sweep). Setting up a merchant account usually takes weeks and requires hours of paperwork to be filled. Most other payment gateways require merchant accounts.

Subscriptions

The subscription side of things is also super easy. Just add a few subscription plans in the stripe control panel and sync the plan ID’s with the ones you are already using on your site. When a customer switches plans stripe does all the annoying prorata stuff for you out of the box (so your customer doesn’t loose money and you don’t have to fiddle around with PayPal partial refunds).

They also have a nice and simple way of getting invoices out the API into your system so you can show customers exactly when what they paid for.

How I Switched in Only 2 Days

The trick here was to keep my infrastructure exactly the same. The only change I made was to the PayPal endpoint (i.e. the PayPal IPN script). At the top of the IPN script I check to see if the request is from PayPal or Stripe. If it’s from stripe I convert Stripe JSON parameters to PayPal IPN POST parameters. Hey presto everything works in an instant.

To get going the only new stuff I needed to add was a credit card form and a cancel button and test, test, test for a day or so.

Nuking Test Data

I was just about to write to the stripe team to suggest it would be awesome if they had a way to nuke all my sandbox test data… but then I wondered if they already had it. In the settings control panel I found a button marked “Clear Test Data”. Simple as that.

Conclusion

Due to the above PayPal IPN trick, ease of testing, ability to erase test data and instant go-live approval I converted my subscription site from PayPal to stripe in only 2 days. I switched it live on day 3. Not too shabby.

I don’t usually rave about this kind of service but they built something in a way that I would have loved to build myself. These guys really thought this through from business to UI to documentation to technology.

Awesome job guys!

Did you hear the one about the Irish ATM?

I had a fantastic trip to Ireland for the new year. During my trip I used an ATM machine to get some cash out. I was surprised by the default cash options. I’ve been racking my brains and can’t seem to come up with any sensible rationale for these amounts. Do you have any idea what the business objectives might be here? (See below pic)

Answers in the comments please!

 

Pluggio Evolving New Design & Mood Board

This is the (evolving) new design. What do you think?

This is the (evolving) mood board. What do you think?

 

 

Maximizing Revenue Through Magic Levers and Three Dimensional Pricing Plans

TL:DR

Conventional wisdom says SaaS apps should have three pricing plans. You should raise prices until you find a price the market can bear and then stick with it… I tried that. It didn’t work out.

I then tried a different experiment. I segmented my site into three separate marketing sites each with its own Plans and Pricing. As a result, I saw signups for higher plans skyrocket.

My Plans and Pricing Sucked

I was lucky enough to be invited to speak at Microconf 2011. On the second day of Microconf, just after I had spoken, I ended up having lunch at a table with Hiten Shah and Ramit Sethi. Our conversation was immediately focused on Pluggio. To my surprise, both Hiten and Ramit felt my Plans and Pricing were completely wrong.

The thinking at the lunch table went like this:

We (Hiten and Ramit) only have one Twitter account yet we are power users with large followings. We can sign up and get huge value from your product at a price point of $9.95. That’s crazy because power users like us should be at a much higher price point! At that time my Plans and Pricing looked like this:

Time for a New “Magic Lever”

As always, I spoke about my problem to my trusted network on StartupGuild and discussed it with Jason Roberts on Techzing. During a few of those discussions some people suggested I needed a new metric. A way to determine a user’s level of success on Twitter, “something like number of followers”. Hmm, but how could I determine someone’s level of success on Twitter? I thought long and hard about it. Hmm. “Something like number of followers”. What could that be?

Holy sh*t! It could be number of followers!

What a great metric! I could make a low price plan for someone with less than 500 followers and high price plan for someone with more than 25,000 followers. And I could give both of them access to the same power features.

After an even deeper look, I discovered a few more possible pricing levers:

  • Number of buffered/scheduled messages
  • Number of automated RSS feeds
  • Grow your following – YES/NO
  • Spreadsheet scheduling – YES/NO
  • Post to Facebook – YES/NO
  • Local Search – YES/NO

Find out What Your Customers Need

During the same Microconf lunch, Hiten and Ramit advised me to use survey.io to get a clearer idea of what my customers need from Pluggio and price accordingly.

The survey results were confusing. One of the most important sections asks – “What is the primary benefit that you have received from Pluggio?” – the answers varied widely. Here are some examples:

  • “Even flow of tweets on my power tweeting account.”
  • “Manage all of my client accounts perfectly!!”
  • “Web based. Nothing to download, therefore nothing to slow my machine down.”

In short, 72 survey responses seemed to describe four completely different products.

Too Many Products, Too Many Levers, Too Few Plans

This left me in the rather unfortunate position of having four products, eight pricing levers, and only four plans to make it all fit.

In another brain storming session with Jason Roberts he suggested the idea of splitting the Pluggio home page by user type, “the way Dell does it”. (When you hit the Dell landing page, you need to specify what kind of user you are: Home, Business, Public or Enterprise. After your initial choice the site remains segmented to your market from that point forward.)

Jason asked if I had any such user types.

“As a matter of fact I do,” I responded. “I have Power Users, Business & Agencies.”

This meant I could split the marketing site into three different perspectives. Even better, I could architect each version of Plans and Pricing to be perfectly relevant for each user type rather than having to knock square pegs into round holes.

Three Dimensional Plans and Pricing

I had an epiphany. If I could think of a way to make it easy for users to flip between each “user type” of Plans and Pricing it could create a new type of vector. User type itself could become a magic lever.

I quickly mocked up a page with three simple links above the Plans and Pricing matrix to make it easy to flip between them.

Now, instead of trying to think of each user type’s Plans and Pricing as its own page, I began to think of three intertwined Plans and Pricing pages as part of a larger three dimensional matrix.

This is what I came up with:

This gives the user a new axis of upgrade possibility. A vertical axis to jump from one user type to another.

The low-end upgrade path has been calculated to move from $9 -> $19 -> $29 -> both horizontally (within a user type) and vertically (jumping between user types). The plans also jump in increments of 10, 20 and 30 in each direction. So which ever way you look at it, it feels like a natural upgrade path.

The Results

The chart below is a breakdown of new Pluggio signups for the past ten months (I introduced the new plans from 9th September onward).

$9 pricepoint $19+ pricepoint Paid Signups
Mar 41% / $298 58% / $429 48
Apr 47% / $298 53% / $349 47
May 49% / $308 51% / $319 45
Jun 40% / $218 60% / $329 38
Jul 36% / $99 64% / $179 20
Aug 36% / $89 64% / $219 33
Sep 1st-8th 44% / $79 56% / $99 12
Sep 9th-30th
15% / $81 85% / $461 28
Oct 31% / $144 69% / $318 30
Nov 18% / $90 82% / $405 27
Dec 12% / $81 88% / $602 27

(Please allow for a small margin of error due to rounding and plans prior to September 9th being $9.95 versus $9.)

Overall revenue is better in recent months, however in earlier months revenue was also good due to the product being shiny and new. The main goal was to get the product out of the revenue slump n July & August which seems to have been achieved.

 

AnyFu Brand “Mood Board”

Here is the “Mood Board” for AnyFu that outlines the basic concept of the brand.

AnyFu Logo (working version)

We’re very happy with this working version of the logo. The orange circle is an asterisk. In coding this is a wildcard and means the same as any.

AnyFu Character Is Here

If you’ve been listening to TechZing you’ll know that I’m starting a new company with Jason Roberts called AnyFu.

We just got our finalized character created by the magical genius of  Scotty.

We’re stoked about this awesome artwork!

The Last Of The “JV Songs”…?

I was rummaging around my Hard Disk and found this song. It’s the last full song that I wrote and recorded before I gave up trying to be a professional musician. It makes me think that one day, if I can get some free time, I’ll make “that album” that I’ve always wanted to make. One day…

Shine (Needs a decent set of headphones to be heard as intended.)

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Download