From Friday, May 13, until Sunday evening, I barely left my computer. I was working, you see. My assignment: to sprint through id Software's brilliant reboot of Doom, only peeling myself from my chair to fix snacks and answer nature's call.
Sounds like a dream job, right?
Yes, and no.
Five days earlier, Bethesda had issued a statement to press outlets informing us that we would not be receiving review copies until launch day. There was no review embargo; we were free to publish reviews as soon as they were ready. As excited as I was to crack into the game, I was also irritated. The original Doom is a timeless classic, one I revisit at least once a year. I'd volunteered to review "Doom 2016" for Shacknews out of a genuine interest to see if this reboot measured up to its canonized forerunner.
But Bethesda's decision to hold review copies until launch day moved the goal posts. I was no longer planning to spend nearly every waking moment at my computer because I wanted to. Instead, I'd have to.
I don't want to give the impression that I'm complaining about having to play a video game. There are worse jobs. Virtually every other job is worse. But Bethesda's non-embargo embargo got me thinking about review restrictions—how they hurt consumers and critics, and how they can help when done right.
Timing is Everything
Bethesda and id Software cordially invited players to dust off their super shotguns and return to hell on Friday the 13th of May the most apropos date imaginable for a game like Doom. That Monday, critics were fidgety. Less than five days remained until launch. That left one work week in which to digest what was shaping up to be a sizeable game if we wanted to have reviews ready to go for launch.
That same day, Bethesda's announcement came through. "DOOM is a robust game comprised of a single player campaign, online multiplayer, and SnapMap, and should be experienced as a complete package. As DOOM's SnapMap and multiplayer modes both require access to a server that won't be live prior to launch, review copies will arrive on launch day."
The explanation raised eyebrows. Sometimes publishers are able to flip a switch, so to speak, and let critics take the maiden voyage on a game's servers. But even if that wasn't the case with Doom, and setting aside Bethesda's Complete Package logic, there was no good technical reason why we couldn't begin "cracking on" with the campaign, as Jim Sterling put it on Twitter, to get a head start on reviews.
Polygon agreed, and asked Bethesda for permission to play the campaign early. Bethesda shot them down.
Most outlets don't worry about beating specific competitors to the punch. I wasn't willing a download code to appear in my inbox so I could scoop GameSpot or Kotaku or IGN. Ideally, reviewing a game isn't a race, because in most cases the date we receive access to a game matters more than when we're allowed to post reviews. A week out from release, preferably two or three, gives us time to soak up an experience and craft eloquent critiques that cover all the bases our audiences want to know about. When an embargo lifts, reviews can go live at the same time, putting outlets on more or less equal footing.
Holding out on review copies until Doom became available for purchase forced critics into a tough spot: fly through the game's content to get a review published within the first few days of its availability, or move slower and more carefully, and hope audiences still care what you have to say later on.
I'm not trying to exaggerate our situation. No jobs were on the line; no gun was pointed at my head as I played. But you have to understand the importance of being among the first (or the first) to review a game as high-profile as Doom.
Download codes for the game appeared in inboxes late Thursday night and early Friday morning, and reviews-in-progress sprouted hours later—placeholders that say, in effect, "We know you want a number, and we don't have one right now, but if you come back soon, we will. Promise!"
GameSpot's Peter Brown logged six hours in the single-player campaign before sharing his positive impressions. Zack Furniss of Destructoid played until 2:00 a.m., got some sleep, and then put in another hour before publishing his write-up, pledging to "finish this [campaign] up quick" before moving on to multiplayer and SnapMap. PC World's Hayden Dingman wrote his review-in-progress, and opted to keep playing even though he confessed (probably only half-joking) to needing sleep.
At that point, impressions were overwhelmingly positive. Critics seemed willing to lose sleep to play. That's the draw of a great game. I too fell under Doom's spell, and raved about its campaign to my wife, who lent an attentive ear every time I emerged from my office.
But set aside for a moment the praise Doom was getting at that early juncture, and consider the fact that I and other critics were treating the game as a sprint instead of a marathon. Every time I got up to refill my glass, grab a bite to eat, or do anything that took me away from the computer for more than an hour or two, I felt guilty and anxious. I hated that duality, the conundrum of loving a game even as I felt obligated to push through it.
Second, and more seriously, rushing is anathema to Doom's campaign. Its intricate maps encourage players to explore, an integral component of the publisher's vaunted Complete Package: unlike most other first-person shooters released during the past 10 years, Doom doesn't nag at you to hurry up or punish you for venturing off course.
Bethesda tacitly incited this behavior, harmful in two ways. First, critics try to put themselves in the consumer's shoes when reviewing a game. Some consumers plant themselves in front of a new game and finish it in a single sitting, but even when that's the case, they don't feel harried. They're not in a race.
But I—as well as some of my peers, judging by how quickly they were moving through the game—felt compelled to play as fast as possible in order to finalize our reviews and vomit out impressions, balancing expediency with our professional obligation to examine products with a discerning eye. For the first eight levels, I did everything: explored every nook and cranny, checked off every secret, claimed every weapon upgrade point. After that, feeling crunched for time, I switched gears, barreling through remaining missions so I could beat the campaign and spend time on the other two modes.
Charging through a game like Doom is like allocating one day to tour Disney World. Except you have to, because if you don't, someone else will hug Anna and Elsa first.
The Rumor Mill
There's a reason Doom's campaign turned so many heads, and not just because it's magnificent.
Popular consensus says that early reviews translate to publisher confidence, while late ones are tantamount to bunkers where publishers can take cover when critics and consumers drop nukes on total duds.
That's not always the case. Erik Kain of Forbes.com pointed out that Ubisoft has a habit of stretching embargoes until the last minute. Child of Light, Rayman Legends, and South Park: The Stick of Truth had launch-day embargoes and garnered reviews on the high end of the scale.
Still, the point is that passing out review copies early indicates confidence, another strike against Doom as it rounded the corner to May 13th. Its multiplayer beta from earlier in the year made players recoil due to its blandness, and handing out review codes to the press on the day of release, rather than well beforehand, seemed an implicit admission of guilt: Doom was going to be a disaster.
But it wasn't. Sure, SnapMap can only dig so deep, and the multiplayer is competent if flavorless, but the campaign is a masterclass of new-meets-classic design, and carried the weight of the Complete Package. Bethesda's decision to build a wall between critics and early review copies constituted poor marketing, plain and simple, and did a great game a disservice.
Feeding the Machine
Giant Bomb's Jeff Gerstmann pointed out that harvesting a bountiful crop of high scores on day one lets publishers shoot their game to the top of Metacritic's charts, and gives them occasion to regale investors and boost or stabilize stock prices. Not all releases have endings worth of a Disney animated film.
Ben Kuchera made waves in 2014 when he wrote a piece for Polygon accusing Ubisoft of "weaponizing" embargoes by restricting reviews of Assassin's Creed Unity until noon on November 11, the day the game launched—thereby tying critics' hands so they were unable to warn consumers that the game was an unmitigated disaster, and one of the most broken high-profile games of the then-new generation of game consoles.
Flouting such transgressive restrictions is an option, but not a viable one. Breaking an embargo usually results in a blacklist, barring offending outlets from review copies and bleeding them of page views. Bethesda's non-embargo embargo on Doom was just as bad. Either the publisher lacked confidence in id Software's product, or it evinced a blatant disregard for the service critics provide.
Publishers are in the enviable position of being able to play the field. They believe they no longer need critics, at least not like they used to when magazines and primeval online review outlets were the only sources of feedback for consumers. They negotiate fetching pre-order bonuses like early access to content and exclusive in-game items so that if a game bombs commercially, they still manage to cash in a few million before word gets out.
Streaming platforms collectively represent another alternative for publishers—some captained by independent streamers, and other hosted by publishers, circumventing middle men to shill directly to consumers.
There are objective streamers. However, streaming platforms like YouTube and Twitch are still the Wild West of game reviews. The shackles on these content creators are often looser than those that bind critics. For instance, popular streamers can post a game review hours, days, even weeks after press outlets like Shacknews because their audience will watch whatever they make, whenever they decide to make it. Their views are based on content, not on reviews specifically.
And streamers may be able to skirt review guidelines on the basis of semantics. Onerous restrictions like Konami prohibiting reviewers from disclosing the length of cutscenes in Metal Gear Solid 4 (an actual, honest-to-goodness mandate) can be evaded by creating a "Let's Play," videos where streamers talk freely without stamping on a score, instead of a "review," considered an official critique. Context matters. Furthermore, "Let's Play" creators can get away with not finishing the game, a condition publishers and consumers expect of critics.
User reviews are, at best, untrustworthy, and meant to be taken with a mountain of salt. On May 13th, one Steam user logged 1.4 hours in Doom before writing "Holy !@#$ bro! 10/10!! Bro!" Not everyone was as pleased: "wtf... the game feels cheap everywhere. The music, opening, and playthrough. I want my 60 bucks back," wrote one disgruntled player after less than two hours.
Salty and sugary feedback from users blended into hype soup. By late Friday, Doom sat at a 91% aggregate, translating to "Very Positive" on Steam's scale. Per Valve's guidelines, users can leave a review for any game they've launched—not only games they've purchased, but games played for free during special promotional weekends, and through the platform's Family Sharing option.
Doom's opening sequence is, dare I say it, even more action-packed than the first couple of rooms of the original's first map, E1M1. But getting through the first 30 to 60 minutes isn't enough to warrant rabid opinions in either direction.
Give and Take
There's a mistaken belief that publishers and game critics enjoy a symbiotic relationship: they scratch our backs with free swag and invites to fancy parties, and we return the favor by rocketing their games to the top of Metacritic's vaunted charts. Sometimes those are the results, but rarely are the motivations of either party so nefarious.
Critics are glad to hype games that deserve to have their praises sung. However, critics are also the first line of defense between consumers and bad games, and that often puts them at odds with publishers.
The fact is, traditional game critics are a dying breed. YouTube was our extinction event, leaving us fossilized beneath layers of Let's Plays, user reviews, and Kappa emoticons. Many widely read review outlets that appeared strong on the surface have fallen, and even the top dogs fight for scraps left by PewDiePie and Total Biscuit.
But we're not all gone. Those of us still standing are here because we care. We care about games, and about crafting well-informed and well-written opinions about them. Publishers should care that much more about their products, even if it means accepting that game reviewers are not always on their side.
The following resources were helpful in writing this article: reviews and reviews-in-progress from IGN, Kotaku, GameSpot, PC World, VG 24/7, Destructoid; user reviews dissected on Kotaku; social media posts from Twitter and Reddit; editorials and blogs concerning embargoes from Forbes, Polygon, Jeff Gerstmann, and a couple of pieces from Kotaku; and my own review on Shacknews.
David Craddock posted a new article, Opinion: Doom Release Highlights Shifting Problem of Embargoes
They claimed it was because of the online component of snap map and stuff that wouldn't go live until release date, but they could have locked that out and just allowed single player before hand. I mean, they knew it was good and must of known it would review good.
Several critics made that point. As I mentioned in the article, Polygon asked to review the campaign separately and was turned down.
To be honest, Bethesda's "Doom is a total package and should be experienced as such" made zero sense. The game doesn't feel cohesive; selecting Multiplayer or SnapMap from the menu prompts the game to reboot, leaving you feeling like you launched a completely separate program--because you did.
A couple of problems with this article:
- In one paragraph, you explain that releasing reviews isn't a race and that you aren't competing with other outlets. In the next two paragraphs, you then explain that being one of the first to release a review is essential.
- You state that you can create Steam user reviews without owning the game(s) you're reviewing. This is inaccurate. You have to both own the game on your Steam account and actually play it before you can post a review for it.
Thanks for your comment. I'll address your points individually:
1) You didn't mention specific paragraphs, so I'm guessing, but what I meant was that under normal circumstances, there is no competition between review outlets. In most circumstances, critics receive copies of a game a week out, sometimes more, and are allowed to post reviews on a given date. Since most critics get their review copies on or around the same time, they have plenty of time to finish the game, write their review, and schedule it for the minute an embargo lifts.
But in this instance, review copies were scattered, so everyone felt pressured--not to compete directly, but to publish their review within the short window during which reviews are relevant. It wasn't a case of, "I've got to finish this game and publish my review so I can scoop <Outlet X>!" but rather, "I've got to finish this game and publish my review before so much time passes that no one cares."
My "it's a sprint instead of a marathon" alluded to the fact that everyone reviewing the game--everyone who needed to get a review up in a timely fashion--felt compelled to put the pedal to the metal. Obviously some outlets that don't rely heavily on reviews for traffic were exceptions. Giant Bomb, for instance, doesn't seem to count reviews as one of their bread-and-butter content offerings, so I'd imagine Brad Shoemaker, who reviewed the game, probably wasn't feeling too pressured, though I wouldn't presume to speak for him.
2) Thanks for pointing this out! This was faulty research on my point. I'll fix this and push a revision live.
If you're interested, I double-checked my research and found that I was both right and wrong.
Per Valve's guidelines for Steam user reviews, the only requirement for leaving a user review is launching a game. Meaning, players can leave reviews for games they've played through Family Share and free weekends -- both cases where purchase is not necessary.
Just thought you'd find that interesting! Here are the guidelines if you'd like more info: http://store.steampowered.com/reviews/
It sure seems that way. It also seems (and this is purely conjecture on my part) that Bethesda had a firm grip on the reins: id developed Doom, but Bethesda called the shots on all aspects of its publishing and marketing. Again, just a guess based on who controlled messaging, press releases, etc.
That's good to know.
It also sounded like Bethesda was more involved than just marketing the game.
Very cool. I'll have to listen to that episode. I tried Idle Thumbs years ago but bounced off it. It seemed like nothing but inside jokes and all the hosts gasping for air at said jokes. Had no idea what was going on.
I'd probably enjoy it more if I'd listen from the beginning, but I have no interest in doing that. Oh, well!
I guess this creates an interesting topic for discussion, but for the most part I feel like you're preaching to the choir.
I hoped to generate discussion. Plus, I'm a writer, and when I want to discuss something, I write down my thoughts to share with others.
Everything I saw and read up to the release of Doom was pointing to an unmitigated titanic atom bomb of suck. Multiplayer beta did more harm than good and holding back reviews until release screamed in my mind UH OH. I'm happy to say the single player can more than hold its own BUT the unconfident actions on Bethesda's marketing and review process definitely hurt my purchasing decision. It's not until people I know and respect actually got to play and submit their opinions that I decided to purchase it.
Good to get some behind-the-scenes insight
Glad you're on the Shack team. Good write-up of what happened behind the scenes. I share many, if not all, of your sentiment about embargoes. Publishers just gotta man-up on this front. Do you have confidence in the product, or not? If not, then it's on you for letting it get through development and you didn't do enough to deliver a quality product. Not every game is going to be a 10/10, but publishers cancel enough stuff that coulda been great, so it's not like they couldn't see the forest from the trees if that's the case.