ImRage: Why You Can’t Trust Big Review Sites

If there’s one thing that’s downright poor in the world of gaming journalism it’s the way the games are rated. You log onto the most popular gaming websites like IGN or Gamespot and you see swaths of games rated 7, 8 or 9 and, less commonly, a 6 or a 5. Only the worst of dysfunctional unplayable muck get scores below 5 and if 5 is supposed to be an average, then how is it even possible that so many games are above average? (It doesn’t even make intuitive sense if you think about it!) I’m not going to go into the details of why I think rating games based on a number out of 10 is a poor system because many others have tackled that particular issue before me. Instead, this week, I’m going to talk about something a little closer to home: why you can’t trust the big-name rating sites out there.

Picture this: You go the store, you pick Syndicate off the shelf and you pop onto your phone to check your favourite review site for a score. Maybe Gamespot: 7.5/10. Seventy-Five Percent? If we’re going with a standard rating scale (like the ones used to grade you in school or university) then 75% is excellent; at 75% you should be expecting the cream of the crop. I mean if a movie got rated 75% on Rotten Tomatoes I doubt you’d be skeptical about going to watch it. Too bad that with Syndicate, this isn’t the case. For anyone with reasonable quality standards, Syndicate is decent but unimpressive. It’s not something you rush home to play; it’s entirely missable and it’s doubtful that it will be remembered outside of 2012. In fact, many readers of this column probably haven’t even heard of it and if you haven’t, don’t bother looking it up. If you want a good FPS, there are so many better options and, frankly, 75% gives you the impression of Four-Star Cuisine when what you’re getting is McDonald’s.

You see, the problem with video games is that as much as we want to compare them to movies, music and books, they are not these things and they never will be. A movie costs under R50 and even if you go see something mediocre with your friends, it’s not a complete waste if you have a good night out and find something to laugh about; at worst, you’ve lost two hours and the price of a take-out meal. A couple of songs on iTunes are hardly that pricey either and you have the option of previewing them fully on YouTube before you buy them. A book is maybe a little different. If you buy something you don’t like, it’s R100 down the drain, but book reviews tend to be of a much higher quality and most people don’t seem to read much these days anyway.

Video gaming on the other hand, is a huge investment. First you need to put a couple of thousands into a gaming console and TV set or a decent PC and then each game you buy costs in the region of R500 to R700. Even if you make use of the second-hand market and trade-ins, it’s a hobby that costs several orders of magnitude higher than most and if you’re putting that much money into it, you really should be having a hell of a time. If you’re not, then you might as well just go and spend money on things you’ll enjoy more; like travelling or going out with your friends. There are few feelings worse than paying R600 for a game that you really don’t enjoy and then forcing yourself to play through it because you don’t want your money to go to waste; trust me, I’ve been there.

That’s why I feel that gaming journalists have more responsibility than any other media reviewers in making sure that they correctly evaluate games. Few people are rich enough to afford buying crap games and, especially in South Africa where the prices are high and the incomes are low, gaming is a huge luxury. I guess that’s why it saddens me that gaming journalists do the worst job out of anyone; they’re essentially an extension of the advertising. In fact, that’s pretty much their function. We’ve all heard the story about the GameSpot editor who got fired for rating Kane & Lynch 6/10 (click here) and when Eidos offered to let magazines review Batman: Arkham Asylum early if they scored it 9/10 or higher (click here). It would be nice if these were just isolated occurrences but if you take one look at the image I posted up top, then the inflated review scores should tell you that a lot more than a few games are getting higher scores than they deserve.

And if you think about it for a second, it’s not all that far-fetched, it’s simply a case of misaligned incentives. Reviewing websites make their money primarily off of advertising and, as such, getting website hits are very much on the top of their priority list. One of the best ways to do this, of course, is to review as many games as possible and to review them early so that they can get attention ahead of other sites. Video game developers, on the other hand, make their money by selling as many games as possible and if a popular site is going to rate them highly and recommend them to consumers then this fits right into their plans. What you have then is a market where reviewer sites are trying to please developers so they can get the games first and developers forming relationships with popular sites where they provide games and the sites rate them highly.

If you’ve studied markets with conflicts of interest, like the pharmaceutical industry (click here and here), then you know that this is a very real issue in the world and what this results in is a market where developers and the suppliers look out for each other’s interests and nobody looks out for the interests of the consumers. For your information, that’s us by the way. What’s scary about this, is how much power developers actually have in the industry. Few will complain when mediocre games like FF13 are rated 82% on Metacritic (any game in which you have to play through 20 hours of crap to get to the ‘good parts’ is mediocre) but when Medal of Honour: Warfighter was rated badly recently the developers actually publicly complained about it (click here)! That’s why it’s been positive in the last year to see gamers rally against companies like Capcom and their abhorrent DLC practices. It’s also been encouraging recently to see that gamers are starting to become more and more aware of how untrustworthy the big name review sites can be and that needs to continue.

It surely must be a rude awakening to many to realize that big name review sites aren’t your friends but, in the long run, it really will benefit you as a consumer. Gaming is costly as a hobby and it requires roughly R600 and 8 hours to enjoy a single game; both of which are valuable resources. So the next time you pass a Medal of Honor game on the shelf and it’s rated 9/10, you should think very carefully about whose recommendations you’re going to trust. Maybe ask your friends or watch Zero Punctuation… that guy usually gets it right.

See you in two weeks…