"When I just see a score, whether it's a Metacritic score or 5 stars or 4 thumbs, that doesn't tell me anything," Olin told Shacknews during an extensive interview, to be published in full at a later date. "I am never surprised when there's as much as a 40% or 50% variance between Metacritic numbers and user numbers."
"My pet peeve is that game reviewers are lazy," he said. "Not all, but in terms of the reviews [something like] 'This game isn't as good because let's compare it to that game over there and that game was great.' Who gives a, you know, bleep?"
"How can you review a game, how can you give a comment about a game like Grand Theft Auto IV, that has 40-plus hours or more of gameplay, if you've only spent 2 1/2 to 3 hours playing it," Olin asked, describing his query as a "challenge" to the industry.
"It would be like reviewing a movie but only seeing the opening, first reel. I don't think that's fair, or is it accurate," he explained.
Prior to his appointment as AIAS president in 2004, Olin served as the vice president of marketing for publisher Eidos and in a position at Microprose. His official biography claims he was "instrumental in raising consumer awareness of well-known videogame publishers and properties from Nintendo to Sega Genesis."
Olin stressed his view that there are "a lot of game critics, but very little critical analysis," describing a "meaningful" review that one that gives "a point of view...good coverage as to what the game maker was trying to do, how they were trying to involve you."
In that instance, he explained, "it's only fair to also point out in the review that some things work better than others, there's some things that were disappointing or didn't live up to the premise or the promise."
However, Olin was clear that "some games obviously don't require [critics to complete the game]," listing Madden as one such example.
I usually look at the score and the summary paragraph at the end of the review and maybe skim up to see some of the points that piqued my interest in the summary. also like watching the video reviews. not sure how this guy knows they only spend 2-3 hours on playing the game
When you have a strict deadline to meet, it happens.
Most sites only get a copy of the game a few days before the release, and the review is up the day of. Can they really play through the entire game and still write a 3-8 page review of game in such a short period?
I was the senior editor of a major gaming news and reviews site for three years. There is NO area of the media which faces more deadline pressure than the game review; even movie reviews have more leeway, as the movies always debut late in the week. With game reviews, the pressure is to have the review posted at the same time or earlier than the game is available in stores. With the millions of review sites out there, not every one can be on the press list for advance review copies; as a result, I'm willing to bet that a large number of game "reviews" out on the 'net are outright fakes, based on leaked betas or pirated copies of the game. The deadline pressure is so intense (we in the the biz call this a zero-deadline article, or "runs when it is done").
Magazine (print) reviews have to be written 3-6 weeks before the magazine goes to print, minimum; as a result, those reviews are often based on the last beta before Gold, and now the bigger magazines have taken to flying journalists in to the HQ of the publisher or developer and locking them in the room with a console and a copy of the game. The MGS4 reviews were done this way, right in SOA HQ. This is the equivalent of being "wined and dined" by the industry, and the urge to bump your numbers to the positive cannot be ignored, especially when they buy you a nice big steak and cover your airfare in business class. Their capital investment in the journalist of $4000 or so for the review junket is a tiny amount compared to what that 9 or 10 score (which MGS received almost universally, an undeserved score) can do for sales. I agree with what Olin says here, but the journalists themselves are not to blame; you've got a system and way of doing business that is fundamentally flawed, and that drives the irresponsible behavior we see on the scoring and genuine critical analysis side.