The Problem With User Reviews

The vast majority of people who submit user reviews are motivated by an experience they loved, or one that they hated. That’s why when I go to Yelp, or Amazon, I’m usually either looking at very positive reviews or very negative ones, to the point that sometimes it’s hard to believe that each review is about the same place or product. As a user, this actually complicates my ability to make a decision.

What do you do when you have a lot of biased people submitting their opinons? How do you manage a bunch of random comments and meaningless rants in order to make the content useful to others?

Yelp is quickly becoming the go-to place for online reviews. But the bigger it becomes, the harder it is to use. Here are three reviews that I recently had to compare. In browse-mode, I’m skimming a couple keywords and glancing at the star ratings. Then I start to get confused and wonder why one person hated it and the other loved it. Who are these people and why should I trust them? Does knowing how many reviews they’ve written make me trust them? Not really. Users who’ve written over 200 reviews are long-time Yelp users, and perhaps restaurant afficiandos who really like sharing their opinions – but, if I wanted an expert review, I’d be using a different site. What really matters is finding the opinions of people with similar tastes, similar interests…people who are most like me.

This isn’t a new idea. At Yahoo! Travel, we talk about surfacing the opinions of “travelers like you” – people who also enjoy skydiving, or beaches, or luxury hotels. But this idea applies to all opinions. When someone you know gives you an opinion, you can temper that opinion with what you know about that person, and how similiar that person is to you. I always like to give the example of movies: A movie that is rated very highly by someone who I know has awful taste in movies actually means that I will avoid that movie at all costs.

Instead of how many reviews the reveiwer has written, it would be more useful to see snippets showing what other restaurants the reviewer really liked and really hated. Ideally, however, Yelp would match my profile up with similar profiles (similar ratings for the same restaurant, similar interests listed, etc.) and then prominently surface the opinions of those people as I browsed the site. Taking this a step further, of course, I’d want easiest access to opinions of people I know.

Sites that host user reviews will continue to struggle with this problem as the sheer number of submitted opinions begins to kill the usefulness of aggregate ratings. If you take the average of a bunch of 1-star ratings and a bunch of 5-star ratings, you get a 3-star rating, and that doesn’t say much. At the same time, a user who has to spend a lot of time reading and evaluating several opposing reviews isn’t getting a great experience either. The essential problem lies in allowing the user to quickly identify the reviews that matter the most to them, for that particular place or product.