The vast majority of people who submit user reviews are motivated by an experience they loved, or one that they hated. That’s why when I go to Yelp, or Amazon, I’m usually either looking at very positive reviews or very negative ones, to the point that sometimes it’s hard to believe that each review is about the same place or product. As a user, this actually complicates my ability to make a decision.
What do you do when you have a lot of biased people submitting their opinons? How do you manage a bunch of random comments and meaningless rants in order to make the content useful to others?
Yelp is quickly becoming the go-to place for online reviews. But the bigger it becomes, the harder it is to use. Here are three reviews that I recently had to compare. In browse-mode, I’m skimming a couple keywords and glancing at the star ratings. Then I start to get confused and wonder why one person hated it and the other loved it. Who are these people and why should I trust them? Does knowing how many reviews they’ve written make me trust them? Not really. Users who’ve written over 200 reviews are long-time Yelp users, and perhaps restaurant afficiandos who really like sharing their opinions - but, if I wanted an expert review, I’d be using a different site. What really matters is finding the opinions of people with similar tastes, similar interests…people who are most like me.
This isn’t a new idea. At Yahoo! Travel, we talk about surfacing the opinions of “travelers like you” - people who also enjoy skydiving, or beaches, or luxury hotels. But this idea applies to all opinions. When someone you know gives you an opinion, you can temper that opinion with what you know about that person, and how similiar that person is to you. I always like to give the example of movies: A movie that is rated very highly by someone who I know has awful taste in movies actually means that I will avoid that movie at all costs.
Instead of how many reviews the reveiwer has written, it would be more useful to see snippets showing what other restaurants the reviewer really liked and really hated. Ideally, however, Yelp would match my profile up with similar profiles (similar ratings for the same restaurant, similar interests listed, etc.) and then prominently surface the opinions of those people as I browsed the site. Taking this a step further, of course, I’d want easiest access to opinions of people I know.
Sites that host user reviews will continue to struggle with this problem as the sheer number of submitted opinions begins to kill the usefulness of aggregate ratings. If you take the average of a bunch of 1-star ratings and a bunch of 5-star ratings, you get a 3-star rating, and that doesn’t say much. At the same time, a user who has to spend a lot of time reading and evaluating several opposing reviews isn’t getting a great experience either. The essential problem lies in allowing the user to quickly identify the reviews that matter the most to them, for that particular place or product.
May 25th, 2007 at 4:46 am
Nice post. Over the years, I’ve seen folks try and tackle this in a couple of different ways. Through a very simple, demographic, “people like me” filter (age, gender), through clustering likeminded reviewers based on rating history, and via self selected networks filters. None of these approaches quite get there…
May 25th, 2007 at 12:19 pm
I did forget to mention NetFlix - which shows both an overall rating, and a rating based just on raters like you, for each movie. I’ve found this pretty helpful. I’ve yet to encounter web reviews that can be filtered by “people like me” or any of the other ways you mention - could you give some examples? I do agree with RateItAll’s value proposition, though, that having access to raters who you know or have approved is the ideal case.
May 28th, 2007 at 10:12 pm
[...] Context When we receive information from people we know, we have a lot more context for framing that information, than we do from someone we don’t know. I can temper any opinions I receive from people I know with my knowledge of their perspective, which is obviously harder to do when you don’t know the person sharing the opinion - see my post about user reviews on Yelp. At the same time, just because you know someone, doesn’t mean you share the same tastes in everything, in fact it simply means you are more aware of the ways in which you are similar and dissimilar from that person. [...]
August 11th, 2007 at 12:25 pm
User reviews are really only helpful if they manage to give a balanced view. Ideally you want to know all about the bad aspects as well as the good. You’re already mostly sold on the good as you’re doing a search for the review