Statistical Anomalies

[Demesnedenoir]

Prologue: I was contacted at the last minute to write a best of Fantasy of 2020 article/blog/mailing, or whatever you wanna call it, and agreed if I could use the criteria of award winners, focusing on major Indie Awards. While doing so I investigated the books on both Amazon and Goodreads. That’s when I noticed something that looks really freaking bizarre with reviews. So, here we go:

Thorn by Anna Burke — Amazon Ratings 100/ 4.7 average – Goodreads 690 Ratings / 4.18 Average

The average change isn’t too shocking, BUT it is glaring. A 4.7 average at 100 reviews on Amazon is impressive as hell. What struck me most was the ratings differential. Damned near 7x the ratings on Goodreads. Now, I’d expect a higher number on Goodreads with a lower average, don’t get me wrong, but for comparison, Eve of Snows is less than 2:1. So then...

Jack of Thorns by Amelia Faulkner – Amazon Ratings 19/4.4 average – Goodreads 865 Ratings/3.91 Average

If the first one is a statistical “Huh?” then the second one is borderline statistically impossible. 19:865, 45.5 reviews on Goodreads for every review on Amazon? 6.9:1 is shocking, but 45.5:1? I don’t believe either of these numbers are legit in one direction another. So, for comparison, I looked at a highly regarded book, award winner, and Kirkus best of 2019.

Things They Buried — Amazon Ratings 17/4.8 average — Goodreads 33 Ratings/4.48 average

This is way more like I’d expect, except the lack of ratings in general was shocking. Quickie comparisons... Malik’s Dragons Trail is less than 3:1.

Fatemarked: Over 1k reviews on Amazon, and over 4k Goodreads. Climbing, but this book’s review numbers are on another level.

JA Andrews’ Dragons Reach — 180:191, another statistical anomaly, but in the realm of believability considering she probably has an Advance reader group with overlap, and I’d expect the goodreads # to climb faster as the book is out longer. Looking at her Threat of Shadows, it is 245:699, much more in the expected ratio range.

In my quick exploration, I did not find anything like these two books for statistical difference. Of course they could just be incredible networkers on Goodreads (but if they’re that good at smoozing reviews, why isn’t the average higher? heh heh) or they just don’t make sales on Amazon, which is simply implausible.

So, what’re people’s thoughts on how this disparity would happen?

 

[Eduardo Ficaria]

Although that little sample of values is not enough to draw out conclusions, I'd dare say that the nature of those platforms is one of the main drivers for having more or less reviews. I mean, Amazon is a generic store, like a big mall, so I don't find surprising to see an acute lower tendency from buyers to leave reviews of books there. Amazon is a place where you buy something and then you move on to other things (buy and forget kind of thing).

On the other hand, the Goodreads platform is a social network shaped around talking and promoting books, so I find natural that any reader would find more appropiate to leave a review there rather than in Amazon. Goodreads probably is the web-to-go nowadays for any reader who wants to see proper opinions about any book (it is for me, at least), and where reviewers find themselves more at home talking about books. Not that you cannot find good book reviews on Amazon but, again, Goodreads feels more like the proper place for them overall.

As a side note, there's no books/novels or even a comics section in metacritic, the main aggregator of media critic scores of the internet. I understand why is this (if I remember correctly, that site started as an aggregator of videogames review scores), but this detail makes you think about the relevance of books nowadays.

 

[Demesnedenoir]

The conclusion I can draw is that those numbers are peculiar, I went over more than the sample mentioned in the post. That said, social networking is an obvious answer, and so too might be a niche indie bookstore. I have made my hypothesis, but no idea if I’ll get around to testing it. I doubt it’s actual deceit, because the numbers and ratings neither build-up nor destroy ratings, it’s more of a ratings in the middle to drag the average down. There’s a multitude of reasons why people leave eviews on GR instead of Amazon, and Amazon is WAY pickier about who can leave a review. I’m curious, as I tend to get while late at night and sleep deprived, heh heh.

The gizmo inside my head tends to consider it more evidence to ignore ratings on books, movies, etc., they’re better for flashlights, heh heh. And yet, the eye is drawn to those little stars.

Goodreads is owned by Amazon and from what I hear, it’s kind of fading, left on the vine to age even if not rot. But I don’t really pay attention to such things.

 

[Maker of Things Not Kings]

I've noticed Goodreads tends to see a wave of ARC books and promo copies which are reviewed there. The last book I went there to review, I saw at least thirty reviews that contained the disclaimer that the reviewer had received a free copy of the book for review from the publisher or some other giveaway source.

Not sure what Amazon's policy on this is but this is from the Goodreads policy:

• Pre-publication reviews. Many of our members receive advance copies of books to review, either through Goodreads giveaways or another source. We have no way of knowing the exact date that review copies are available. As such, each book is eligible to be reviewed as soon as it appears on the site.

An example would be, Namesake (Fable book #2) by Adrienne Young. I read the first book last month so I was curious. The second book won't be published until March 2021, yet there are 420 ratings and 270 plus reviews on it right now.

I don't recall seeing numbers like that on Amazon itself where reviews are mostly verified purchase reviews after publication, yes?

I'm not sure about Goodreads fading on the vine. For book lovers it is still a popular social networking place for readers and an article I read last year talked of its ability to drive buyers to Amazon more than any other outside (though yes, Amazon owned) source. I've noticed that quite a few authors maintain active interactions with discussions, book notes, blog posts etc.

These days I never go to Amazon for books/reviews/browsing. But I buy through Amazon and read samples almost daily through Kindle. But my actual book searching, browsing time spent on Amazon itself is now nearly zero.

 

[Demesnedenoir]

ARC reviews are doable on Amazon, they just want it stated as you noted.

Goodreads is also much more gameable than Amazon... GR = an email address, while Amazon requires an account that has spent a minimum amount of money on Amazon to be able to review, hence several readers in Canada and Australia who spend almost nothing on Amazon can’t leave reviews there. Though I’m not making the claim that gaming is the reason for this statistical oddity. ARCs are possible explanations... it’s also a good guess as to why Andrews has nearly equal numbers in both Amazon and GR is ARCs. My guess is they’re in tight with a reader community on GR and handed out ARCs, but people who have an established following of readers are going to be better able to get review/ratings from handing out books. People are oddly averse to leaving reviews. I get contacted by readers often saying great things, and they don’t leave reviews.