How All Movie Review Rating Systems Are Flawed

When Terminator: Dark Fate was released, I tracked for over a week the Rotten Tomatoes ratings

The simple answer to the title: human beings created these rating systems.

The longer answer gets pretty deep. Stay with me as we weigh in..

No matter if it’s the greatest movie reviewer of all time or you’re brand new reviewer dipping a toe in the water: every ratings system you can pick to use — or create on your own — is flawed.

Let me explain what I mean by that and defend why I’m even bothering to use a rating system at all because it’s going to sound very hypocritical.

We use the five star rating system. It’s not the best, it’s not the worst, it’s one of the more familiar systems being used by reviewers. Amazon uses it for everything. People see a score in a five-star review system and understand immediately what it means.

That’s what you want. People to see a rating and say, yes, I understand if the reviewer liked or disliked.

Here’s what you don’t want to see:

Somebody help me understand what the star rating above means?!?!

I considered other rating systems.

Like the grading system (A+ = best, F = worst), but my computer programmer mind kicked in and said that wasn’t a numeric system. I worried that from a programming perspective, I’d have to convert those textual grades to a number at some point like this:

A+ = 14
A = 13
A- = 12
B+ = 11
B = 10
B- = 9
B = 8
C+ = 7
C- = 6
C = 5
D+ = 4
D = 3
D- = 2
F – 1

The grading system gives 14 comparative points which is much better than five stars which, with half stars, only gives 10.

5 stars = 10
4.5 = 9
4 = 8
3.5 =7
3 = 6
2.5 = 5
2 = 4
1.5 = 3
1 = 2
.5 = 1

The other rating system I thought about was the percentage, with 100% being the best and 0% being the worst. That doesn’t need any breakdown, as that gives 100 points of difference for comparison.

Some might be thinking, hey, that’s how Rotten Tomatoes works. They’d be wrong. That’s not how Rotten Tomatoes Tomatometer works.

The % from Rotten Tomatoes is how many gave a positive review vs. a negative one for a movie or TV show. So if the score is 35%, that doesn’t mean the average rating was 35%, it means 35% of everybody who left a review thought it was good.

Wrap your mind around that for a minute. Too very different things. If I’m using a 100% scale and give a movie 35 out of 100. That’s my review score. However, if 100 people are polled and 35 say they loved the movie, those 35 peoples scores are 100%. Now let’s take the other 65 people. Some they loved it (100%), some say they liked it (80%), some say it’s ok, but they still recommend it (70%). The average accumulation of those scores will not be 65%. That’s my point: mathematically and inherently flawed.

But when we see 35% Rotten Tomatoes, we think that is absolutely godawful. It’s a terrible score. The movie must be total crap. It’s possible, maybe even probable, that the score is higher depending on how much the people liked the movie vs. the ones that disliked.

That subjectivity is the flaw and no matter what rating system is used, the same flaws exist.

So, then what is the best rating system? Why not just skip using one and saying: recommended or not?

It’s those movies that sit in the middle, those pesky movies that teeter on the edge of recommended or not are the most vexing to rate. Do you recommend it or not? If you recommend it then you’re putting it in with the very best movies of all time that are also recommended. How do you compare one of the greatest movies with an OK movie that both entertained and have exactly the same score?

Aye, the rub.

So that brings up having more points of comparison in a review. A five star system with half stars is 10 points of comparison. A grading system is 14 points and a percent system provides 100 points. A four star system provides 8 points.

My problem when seeing a 3-star rating is whether or not that’s 3 out of 4 stars (75%) or 3 out of 5 (60%). Big difference in perspective. Not every movie reviewer shares the max rating. We don’t here, but probably should start doing that to make it clearer that we’re using a 5-star rating system, not a 4-star rating system.

At the end of the day those of us reading your review need to understand one simple thing: do you recommend seeing it or not. Yes or no. The why you chose which option is the entertainment angle of the criticism.

Let’s get to criticism. This is where people with opposing opinions sometimes confront each other. If I hate a movie you love, then how does that make you feel about me as a reviewer? Some of you might think my taste is terrible because we disagree. Some might say, hey, he just didn’t care for something I liked, others might call into question my professional standing as a movie critic.

There is an art to writing a great movie review without spoiling the movie. I’m still trying to figure out and learn this craft. It’s easier to write a good critique of a film. I do the latter here all the time, but only TV reviews appear, some of which are spoiler-laden and some are not. The spoiler-laden ones are better in my opinion because they allow me to explain every single detail why I liked or disliked something. It’s harder to do that with a shorter review. How do you criticize what you disliked without ruining the film for those who haven’t seen it yet, especially if that involves the ending?

So, for the spoiler-free reviews, the reviewer has to resort to using vague language. Listing reasons they dislike something, but not being able to fully explain what s/he disliked. Sometimes I can draw that picture in a couple hundred words or less and sometimes I can’t. This is the art of the craft and it’s much harder than it looks.

I could make an argument that a 200 word movie review that doesn’t spoil the film for those who haven’t watched is harder than writing a 5,000 word short story. I’ve done both.

What do you do when others disagree with your reviews? How do you handle that criticism?

I just read the longest ever explanations on how to handle criticism. It’s a good piece if you can get past the wordiness. It’s coming from a highly skilled famous guitar player, Steve Vai. The best part of it is how he personally handles criticism with three words: “I’ll try harder.”

Great response.

“I was doing an interview with the same magazine, and of course, they showed me this article and said, ‘What do you think about this, what he said?’, and I had already kind of practiced allowing him to have that opinion, and what I said was, ‘I’ll try harder’ – the end.

Steve Vai Explains Best Way to Handle Criticism, Recalls Reaction to ‘Very Famous Guitar Player’ Saying He Was Complete Crap | Music News @ Ultimate-Guitar.Com

If someone dislikes your next movie review, just remember to say, “I’ll try harder.” The same can be said for all movie rating systems. We need to try harder.

8 thoughts on “How All Movie Review Rating Systems Are Flawed

  1. Just go convoluted like me – list 20 categories of what you think is important in making a film, such as acting, music, directing etc, and give each a score out of five for a total out of 100. Or just skip scores altogether and the content of the review speak for itself.

    Liked by 3 people

    1. lol, I’ve seen those types of reviews. Complicated might be a more descriptive word 😉 The KISS principle might need to apply. Your system, from a technical perspective, is probably far more accurate and “fair” than a simpler system.

      I remember chuckling when I saw a video reviewer complaining about ratings in the middle of his review and saying, “I gave up on ratings because they all suck.”

      It’s a circle jerk, indeed!

      Liked by 2 people

      1. In the end it all becomes subjective anyway – we like what we like. But I am curious about how I would score films using my own system, and see if the score aligns with my ‘personal feelings score’. I’ve been writing draft posts on this for a while, but who knows if I’ll ever publish them

        Liked by 1 person

      2. So true. Nobody knows better than you what you like. Same for me. I try and qualify when I like some movies, books, games that others don’t , but at the end of the day if it gives us enjoyment, it doesn’t matter what anybody else thinks.

        Liked by 1 person

  2. My principle for movies (and books) is that the top rating is for the elite. In film, Dr. Shivago gets a 10-of-10 (or 5-of-5 on your scale). It combines history, score, acting, story, drama, scenery, characters, and more – the movie has it all! If I give another film ’10’ because “I like it,” then there is no way to express that “I love it, I appreciate it, I learn from it, I would watch it again and again.” This discussion is harder for books, where every author clamors for a top (usually 5-star) rating. I sometimes feel bad that I personally know the author yet “only” offer 4-of-5 stars. Oh! Halves seem silly to me; I prefer 0,1,2,…,9,10 rather than the “exact same” 0,0.5,1.0,1.5,…,4.5,5.

    Liked by 2 people

    1. I like your principle and I’m sort of the same. If you look at how many 4.5 and 5- star movies I’ve rated it’s like around 6%. That’s about in line I’d say with how many great movies I watch out of 100. I realize I could weight the numbers down by seeking out the greatest movies ever made and watching and reviewing those first, but I’m being more organic with how I’m watching/rewatching movies, not trying to seek only/primarily those out.

      As for books, that’s a whole other can of worms.

      As an author myself, these ratings at Amazon make a huge difference in sales. When my novel averaged over 4 stars it got more traffic and love from Amazon and sold more copies. The minute it dipped below 4 stars, sales dropped precipitously. The ratings on my book are not gamed, they are real readers who have bought the book and/or whom when I was out marketing it (that’s a subject), so I understand why author friends of yours pressure you (or make you feel pressured) into giving a 5-star review. A 4-star or single 3-star review, particularly if there aren’t very many reviews, will basically knock the book into no-man’s Amazon land and their sales will tank. And a 1-star review will almost certainly doom momentum unless there are dozens or hundreds (even better) of 5-star reviews to offset the average.

      Liked by 1 person

  3. Great read! Definitely agree with a lot of what you’ve said. So many review systems feel like numbers or ratings pulled out of nowhere with very little context to evaluate what’s going on. It’s why I tried to not only have a clear and defined rating system with qualifications, but posted an article early on talking about how I look at movies and break them down into “metrics”. Like for me a 7/10 is like a C because of years of school drilling that into my head. So when I see someone go a 7/10 is good, I get it because a more liberal use of the scale would probably not keep passing at a 6 or a 7 but maybe a 5. So in that case the 7 is a big deal and is “good”. It’s just my head defaults to ehh, that’s a C. Granted, I think my approach, that’s more in line to the US grading scale, doesn’t explain the difference between movies enough. Like there are 10’s i think are better than other 10’s but my rating can’t demonstrate that. Only my words can.

    Granted, I think that’s the biggest problem with ratings. Numbers and grades are arbitrary because it’s impossible for them not to be. We might agree an element is done well but then the discussion becomes how well and whether or not that element is important to the coherence of the piece itself. That’s why I think reviews are so challenging. It’s not just giving an opinion. It’s almost justifying that opinion in relation to some paradigm so that a reader can evaluate why you think something and see how the point of view develops. It’s something I loved the late Ebert and his reviews. Even if I disagreed I could always get his point of view, and that’s the biggest thing I look for in reviews. It’s also how I feel about review criticism. It’s fine if someone disagrees with a review if they can explain that disagreement in a way that clearly demonstrates their point of view/what they’re looking for/why what they’re looking for is the most important facet/etc. What I don’t like is someone going “I just don’t like it because it’s X” without an explanation of why that X crowds out other elements.

    Spoiler free reviews are so much harder than they’re made out to be and I appreciate you putting it into context. I struggle so much because all my reviews are spoiler free and finding a balance between being ambiguous and sounding like a broken record is hard. It’s always a constant process of revising till it’s clear what you’re talking about and even then it might be not enough or too much depending on the person.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s