My ratings obsession
Anyone who knows me knows that I look at a lot of ratings before I decide to do anything. A good friend of mine once joked I wouldn’t turn left if Yelp didn’t tell me to. Have I taken it too far? – probably – I mean I’m writing a blog post on ratings.
Recently, I was looking for a new book to read when I came across Goodreads. I signed up because it has a recommendation engine that can tell you what you might like based on books you have read and what your friends are reading.
As I browsed the ratings on Goodreads I wondered if they were any different than Amazon’s. What I saw was surprising – books had a very different ratings on Amazon vs Good Reads, even after having hundreds and thousands of ratings. So which one is better?
The Data: Goodreads versus Amazon – Ratings distribution
Based on an index of 10 books equally weighted (weighted towards business books):
These books received 15K reviews for Amazon and 315K reviews for Good Reads
How can thousands of reviews have such a different mean on one site versus the other? And why do Amazon’s ratings have such a “U” shape compared Goodreads?
At first thought you might think – different audiences?
Thanks to Alexa we see it’s not age, education, or family status. The only major difference is that Goodreads skews female more heavily than Amazon. But that doesn’t explain it – why would women on average rate books lower and have a more uniform distribution in ratings. It must have something to do with the ratings system itself. It turns out it probably does (thanks to Sid for helping remind me of the bias in #1):
1) The more work one must do to submit a review, the more extremes you will see in the rating (“U” shape) On Goodreads you simply have to fill out a star rating while on Amazon you have to write a full review. Hence you’ll see many more 1s and 5s on Amazon than what might be true in the real world. So when you look at reviews on Amazon, realize in reality there are a lot of people in the middle who think the books are really 3s and 4s who didn’t review it.
2) There are penalties for dishonesty on Goodreads. Goodreads is a recommendation site. To start getting recommendations you have to rate 20 books you have read. If you aren’t harshly honest about the books you stand the risk of getting bad recommendations.
Amazon, on the other hand, is much more outward facing. In a public review of the book, you may say it was good but it wasn’t for you and settle at 4 stars, but you definitely don’t want to read something like that again so it might get 2 stars on Goodreads.
(Amazon has recommendations too – but these are much more skewed towards browsing history)
All this is to say I think Goodreads reviewers have more skin in the game and are a more representative sample. For now Goodreads = better reads.