Posted in: Games, Video Games | Tagged: arkane studios, Bethesda Softworks, Human Head Studios, prey, Review Policy

Bethesda's Review Process, Prey, And Why We Don't Have A Review Yet

Published Wed, 10 May 2017 17:45:28 -0500

Some games publishers like Bethesda have silly policies where they just don't trust the press. Bethesda is pretty notorious now for only releasing review copies of their games to media outlets less than 24 hours of lead time. This was of course the case with Prey and has led to incomplete reviews, reviews in progress at several sites, and has led to me not posting a damn thing about it.

Bethesda are far from the only ones with such a frustrating review policy, but they are at least equal about that treatment across the board. Several other publishers prioritize which outlets they give keys to first, and being a wee little comics site, Bleeding Cool are often toward the bottom of that list.

I'm not bringing that up just to whine about how unfair my life is as someone who works for a news outlet not necessarily known for their games coverage, but it is a really frustrating practice for reviewers across the board. Any time you get a game with less than 24 hours turn around time, well, you're pretty much guaranteed to be writing that review without finishing the game. Because if you wait to finish it, well, by then no one cares, because your review is crazy late to the party.

So if you've ever wondered why a review doesn't address some game-breaking problem you find in the final act of the game, well, chances are the reviewer didn't have time to get to that point in the game before hitting their review deadline. Giving less than 24 hours for reviews makes sure that harmful leaks don't get out in time to sabotage the game's launch, negative reviews don't hit until it's too late and most people have bought the game already, and reviews are generally not too negative. It's hard to say "this game is the worst thing I've ever played" when you haven't played all of it.

That's why you see so many more middle-of-the-road reviews. Writers are forced to hedge their bets based on only playing a few hours of a game, and I get it. We've all done it at least once in our careers, if not more than that. Sure, some still decide to be incredibly harsh about a game, and indeed some outlets took that path with Prey, while others are prone to lavish praise on games. But the fact remains that these are still partial reviews.

This whole policy basically makes critic reviews worthless. Because we can't do our jobs properly to insure that we've fully looked over every inch of a game before stamping a score on it. A critic's review is intended to be a stamp of authority on the quality of a piece of art. You might read a film review that started with "I walked out 10 minutes in," but that reviewer would have to do a lot to tell you that there was nothing redeemable in the movie's second and third acts based on only 10 minutes of viewing. You certainly wouldn't expect that of a top movie critic, though, would you? If Ebert can sit through North without walking out, well, every film reviewer should sit through the whole movie. And they do.

So why would an game publisher or studio want reviewers to get the equivalent of only the first 10 minutes of their product? It leads to a situation of muddy reviews. It also means that developers can totally change a game around on you in the last act and you'll have no prior warning — because there won't be any reviews out there telling you, "yeah, it's great until you get to the last four hours, and then it's like a whole different game."

With Prey, Bethesda's policy has hurt them even more than usual. It's not only because of those game-breaking PC problems, like the save compatibility bugs that led to several reviewers — myself included — not being able to get more than 20-30 hours into the game before getting locked out. It's also because it's led to a pretty mediocre overall reception.

Kotaku, who have been blacklisted by Bethesda for the last three and a half years, have a pretty stunning rundown of Bethesda's policy, if you'd like some history on that. Their reason for disliking Bethesda's review policy mostly boils down to that time factor. A game like Prey isn't meant to be played in a speed-run. Not for a reviewer, and not for your average player. Sure, you can beat the game in 20 minutes, but that's not the point of Prey. Prey is an atmospheric horror game that really shines when you take the time to look at the details.

Kotaku's Jason Schreier also brings up the impact of Metacritic. Quite a few game publishers have bonuses tied into their Metacritic scores, and those only seem to kick in when games hit a 85 or higher. So they've got plenty of motivation to work with reviewers, streamers, and Youtubers, but Bethesda seems to be adamantly against that.

I'm not going to lie to you, I'm really not looking forward to that review policy when it comes to The Elder Scrolls Online: Morrowind in June. Trying to speed-run a massive MMO expansion? Ha. Not likely, guys.

But at least that's equal, with the exception of those outlets that have been blacklisted. Other publishers like to have favorites, or require you to put in a request for review six months ahead of time — even if you don't have a release date at that point. Quite a few outlets got Mass Effect: Andromeda weeks before I received a key to review it, so my review ended up being over a week later than the average. That's not to mention the fact that we couldn't even get a review copy of the Nintendo Switch — not even on a temporary loan basis. Gavin Sheehan had to actually go out and buy one to review it.

Let's not even get in to the fact that several companies won't give us both a PC and console code for a game because, let's be honest, games don't play the same on PC as they do on console. Like with Prey. We might have had it reviewed by now if we'd gotten an Xbox code as well, so could finish the game and writeup a comparison of how it plays on both platforms. As in, it functions on console, but has some absurd bugs on PC. That kind of thing.

And sure, it isn't just the games industry that does this. Bleeding Cool as an outlet can't get review copies of any Marvel or DC comics. Any reviews we do of Big Two titles end up being because one of our readers bought that issue and decided to review it.

Because critics aren't looked at as necessary anymore. It doesn't pay for companies to work with us — not if we're going to say something negative about them. So DC and Marvel don't work with us, because we aren't their "preferred media partners" and we often get ranked lower on the scale of game reviewers because we're a small site. You can count our current games staff on one hand.

That doesn't mean that we don't get proper time on every title, though. I had more than enough time with Telltale's Guardians of the Galaxy and Square Enix and Koei Tecmo's Dragon Quest Heroes II, but those are definitely outlier situations. The problem is manifold, but it all comes down to a basic takeaway for gamers that many have known for years now — you can't trust critic reviews.

And that is a major problem for the industry. The second you try and get rid of all negative reviews — when you try to control the outcome of what is supposed to be an independent accounting — you've become something much worse than a games publisher.

Enjoyed this? Please share on social media!