Contention Question January 8, 2009
Posted by Mitchell Dyer in "Game Journalism" Industry, Games, reviews.Tags: debate, discussion, reviews, Videogames
trackback
So I spent a good portion of my time at work tonight re-reading the “symposium” that Shawn Elliot spearheaded before the holidays. I’d read it as soon as it was publicly posted, and found it to be one of the most enlightening things I’ve ever read. As someone who’s still up-and-coming in an industry full of equally passionate writers who aspire to reach the same goals, I gobbled it up without really letting a lot of it sink in. I was so anxious that I read it without realizing the full breadth of everyone’s arguments.
I’m not going to talk too much about it right now, but I did want to get some sort of discussion flowing in the comments section below. Whether you’re a reviewer or a reader, I want to see some chatter. I don’t want to pose something a bit more simple than the typical bullshit you’ll see in a forum, like “Why aren’t reviewers objective?” and “Who’s in whose pocket?!” Though I fully expect both of these questions to show up in full sincerity.
So I ask you, simply:
What do you think of videogame reviews?
I’m a part of plenty of forums. Fast Karate for the Gentleman. Official Xbox Magazine. Gamer’s Liberation Front. Obviously, some of these message boards feature more cogent discussion on the subject of games criticism than others, but each feature its own batch of intellectuals with something worthwhile to say. After reading the contrasting vitriolic hatred and envy/admiration regarding writing-about-games in GLF, the attacks on OXM’s credibility, and the constant banter between forum members about how totally sweet Jeff Green is, I wanted to get as many people together as possible in one area to talk back and forth on the subject.
I don’t know how well this forum congregation experiment will work, but I really want to hear feedback on the subject of reviews.
For example, do you read them at all? If so, do you read them, or hound the score? If you do read reviews, what would you like to see done differently, kept the same, or scrapped altogether? Do you have preferred outlets, or others that you completely boycott?
And since it’s inevitable, do you honestly believe that writers can be bought?
Let me know. Reply to other commenters’ musings and thoughts, because I want to know what the general populous think about game reviews, the people behind them, and the industry it supports as a whole. I’ve got my own thoughts, and I’d like to express them somewhere other than a blog entry.
So, let’s talk.
I don’t read them. If I glance over the metacritic page, it’s just to gauge the general reaction of the consumer guide community. The last time I really read and cared about a review was probably almost a decade ago, when IGN told me to buy Tetris DX. On that one, they were right.
“And since it’s inevitable, do you honestly believe that writers can be bought?”
Pretty much man.Happens in any sort of sponsored magazine that reviews content.
I read tons of automotive magazines , and you do tend to see quite a few of letters of people complaining that the magazine slanted the favour towards their sponsors product.
One thing that sticks out to me, and this was before I worked for them, was OXM’s Turok cover last February. The exclusive review was the cover story, and the game received a 7.0 rating of “Good.”
You’ve got to wonder if that PR guy still has a job, what with taking an exclusive with something as “low” as a 7. It’s a decent score, but you usually see these kinds of cover stories netting ridiculously high scores. That was quite shocking.
Obviously, I don’t know how it goes across the entire industry, but Dan Hsu and James Mielke, both former EICs of EGM, have been very vocal about the moneyhat thing as well.
Anyway, I don’t want to dwell on that too much since it’s a sensitive subject that’s quite flammable.
djdrastic: I have heard that allegation before — I think it’s a very common assumption, particularly among people who do not regularly read reviews and therefore don’t have more than one or two examples to draw on when they assume the worst — but what you allege has not been my direct experience. Do you have your own experience to prove or at least support that extremely serious claim? If you’re right, it’s corruption and I’m a thief. If I’m right, it’s libel, and you’re a liar.
To me, “writers are bought off” is the biggest, nastiest, least supported accusation someone can make. Nobody ever has to prove it for everybody to believe it, which is convenient and unfair. All I can do is say “it’s not true, I haven’t been bought off, and I am neither told what to think nor do I tell the writers I’ve empoloyed what to think.” Because that’s been my experience for a decade and a half. The last time I even faced an ethical dilemma like that was in — no joke — 1995. And I took my name off the review after it was sweetened (from a 2 out of 10 to a 5 out of 10…was that really even worth it?). There wasn’t even money involved; I just said, “No, that’s not what I think” and ran my original review somewhere else.
But I’m guessing you still hold to your beliefs that good reviews are bought and sold — whether it stems from a specific disagreement over a review or a series of reviews, or it’s just garden-variety cynicism, I don’t know. But if I say it’s not true, I can’t prove it — nobody can prove something that isn’t true. So it falls to you to prove your allegation as truth. If you can’t, you shouldn’t labor under the belief, and you certainly shouldn’t be spreading around misinformation.
Ultimately, I am extremely replaceable. Even if someone had approached me with an offer, why would I take it, no matter how much it was? Why flush my entire 15-year career down the toilet?
And this is why I still drive a 96 Geo Prizm. Sucks to be honest, huh?
I read game reviews. Anytime I get a new issue of OXM or GI, I always sit down and read every single word in that magazine. Concerning the scores, I usually only make public comments or comments to myself if I’ve played the game and agree/disagree with what they say. When it comes to games I don’t play, they factor in the decision if I buy it or spend money to rent it.
When it comes to the ‘politics’ of reviewing, where some people are paid to give the game a good score, well… I try my hardest not to believe that. I mean, it happens and I know it, but I try to read the review and REALLY see it for what it’s worth. I’m obviously going to make an open comment if I see a really bad game given a perfect 10.
I really don’t know what else to say on this subject. =/
Oh, and letters alleging bias are not proof; that’s proof of reader bias, of the audience expecting to hear one thing but reading another, and refusing to read a publication with an open mind. That’s an whole different can of worms.
After reading what Dan said (because he posted it right when I posted mine, so I only got to read it after I got done typing my words), I can say that I’m so glad that he’s never been ‘bought off’. Like I said, I don’t really believe truly that reviews are bought off everyday, but I can’t help but think there might be that one review somewhere in the world. I’m just glad that Dan could come and confirm his honesty, and I truly respect that (and him as a writer).
UR MY IDOL, BRO. <3
Dan: I want to touch on something you said without getting in to the specifics.
“The last time I even faced an ethical dilemma like that was in — no joke — 1995. And I took my name off the review after it was sweetened (from a 2 out of 10 to a 5 out of 10…was that really even worth it?).”
If you can, could you possibly elaborate on why a score would be juiced like that? I mean, it’s easy to assume that there’s a bag of money if you sign the papers, so I’d like to be a bit clear on that, if we could.
Good to hear you told ‘em to shove it.
Also, it’s worth noting for anyone looking to say “LOOK! It is a corrupt business!” please chill. This is 1995 we’re talking here, and as of right now we don’t even know where Dan was or what he was writing about.
Then I am a fat filthy liar I guess man.It was never directed towards you though.
Go look up Top Gear “Top Gear Magazine Hot Hatch of 2007″ and then go and lookup “Honda Civic Type R Car Review” on youtube.Its like the same holding company drove 2 completely different cars.
I tend to listen to reviews by people I know will like things I like. ZP. Dave and Joel. Rym and Scott. My best friend. I don’t give a shit about the nuances of 7.8 vs 7.9. If I tend to be steered right by a reviewer, then I’ll tend to listen to them more in the future. If they fanboy constantly over one type of game that I hate, I won’t. I don’t get why some people find this so hard a concept, and beg for “objective” reviews.
I use reviews for games I am not sure of or as a way of previewing a game before buying. Sometimes I’ll read them if I just finished a game and I feel like justifying my opinions. Sometimes it’s nice to know you’re not crazy or to have some justification for spending money on it. I don’t read them all the time, I catch them in passing overall.
I usually don’t read reviews of games. I watch trailers and game play videos. As it stands, how much fun a game provides is subjective. But that’s what I care about. So when I do read a review of a game I’m thinking to buy (I don’t think about buying it unless I already like the premise, which is why I own a lot of games about space, robots, and samurai) I usually don’t pay attention to whether or not the reviewer liked it, what I do look for is if they can give me an idea of what actually playing the game is like. If they can do that AND they say they liked it, that’s a good sign. Just saying they liked or hated it, but they talk vaguely about the game and I start looking to see what other people have said.
I’ve tried doing reviews before(not professionally, just more as a “I wonder if I can do this” kind of thing) and this is what I’ve tried to do. I know I might like something, but I know other people might not. And assuming the people I’m speaking towards have an individual mind and can make their own decisions, I try to give them enough to make their own decision of whether they want to play or not.(I don’t know if I actually do well because I think 3 people have ever looked at them, but for what I mean I did http://freegamereview.blogspot.com/ and http://oleetkustudios.net/podcast/ .)
As far as paying off reviewers goes, I’d bring up the Kane and Lynch debacle, but I honestly don’t know anything about it. All I know is that a friend and I tried the co-op and stopped after about 15 minutes because it sucks so much balls.
It pretty save to assume that some (probably very few) people writing reviews are getting paid by a game company (or PR company) to say good things. just like in any other business some people will do anything for money it happens in sales environments all the time a sales person will get a bunch of cash or product from the seller for signing a long term or big contract and they know before hand they will get this money or items.
I don’t read game reviews because I do not know a group of reviewers well enough to know their tastes. I do read movie reviews because I have been following the same guys for years and I know their tastes so I know if they line up with my thoughts or if I need to take what they are saying with a grain of salt.
The most important thing a review can give me is an idea of what I’m going to be doing while I’m playing the game. If it’s Metal Gear Solid 4, I don’t need to know how ridiculous the story is going to be. I need to know if it has multiple axis inversion options, if the control scheme is conducive to the activities I’ll be doing through the course of the game and if not, if the control scheme is at all customizable. If it’s SoulCalibur IV, I don’t need to know that the story mode can be finished in fifteen minutes with a scrub character like Yoda, I need to know that the online is riddled with lag, the options for customizing a character have actually been scaled back from the previous installment, and that game balance in general will need some severe tweaking before the game can even be taken seriously. And for Burnout Paradise, I don’t need to know how immersive DJ Atomica’s blurbs help make the game, I need to know whether I can silence him if I find him annoying without changing the volume of my vehicle’s engine, if I can retry events I just failed, or if I can even cancel events I’m currently participating in that I already know I’m going to fail (for the record, you can’t do any of these things–but you could in previous Burnout games. Progress?).
It’s all about what I’ll be doing and how I’ll be doing it, what impedance I’ll face from bad design decisions, poor programming, or what have you. These are the things that make my controller fly, as opposed to anything having to do with story, setting, themes, graphics, etc. In a well-written review, those kinds of things can have a place, but for what I play games for and what I get out of games, these things are completely tertiary to me, as is an arbitrary number with no meaning or value to me placed on the sum total worth of the game.
That said, I haven’t been reading reviews much anymore. If I do, I’m looking at them after I’ve played a game to see what the person writing the review got out of the game compared to what I did. As for choosing what I spend my time with, I play demos, I rent, and I talk to people whose opinions I understand. I know my own tastes and what I want from a game well enough that trusting someone else’s opinion I’ve never met and never will meet or understand (however valid it may be by being published or on a website) is both counterproductive and a waste of time. I think I remember an ad from a gaming era past, and it’s still a valid quote: “Trust no one. Play it for yourself.” Really, if we aren’t playing for ourselves anyway, who are we playing for?
I’ve seen this question come up many times and my answer is still fundamentally the same. I, as a consumer, have an obligation to both myself and the games industry I love so much to show due dilligence when deciding the games I will invest both time and money into. I get resources from all over (oxm, ign, px360, gamesradar, etc). My key is to learn more about the reviewers and their prefferences so that I have a frame of reference when I read their reviews. I really don’t care about the “how much integrity and independance to reviewers really have” issue, because I cast a fairly wide net when looking for sources. I guess it comes down to, I like adventure games, so if an adventure game isn’t branded as broken, I will probably enjoy it. I don’t really care for FPS’s, but if one is universally hailed as the best thing since sliced bread, I’m going to check it out.
I like to read reviews, and everything pertaining to them. The score is actually the last thing I look at.
What people need to understand is that a review is a single persons opinion and that’s it. There is no right review or wrong review as it’s ALL subjective.
It pisses me off most when people call out a magazine or site for giving a game a score they dont’ agree with..”Site X is wrong because I would totally give game Y a 9 not a 8″……ok well that doesn’t make the site wrong, it just means you liked the game more than the person who reviewed it.
The best way to go about it IMO is to find a reviewer and/or fellow forum poster whos taste is similar to yours and trust their opinion. Look at it like movies, you might have a friend that doesn’t like over the top movies such as Shoot em Up or Wanted but since you do…..their opinion might not resonate with your taste…..does it make them wrong…no, it just means it wasn’t for them.
So read multiple reviews and decide based on your own taste what you should do. In the end it’s what YOU think that matters…if you’re happy witht he game, whoooooo gives a shit on what other people think?
DJDrastic: Dan wasn’t trying to be condescending. Trust me. I’ve spoken with the guy enough to know that that was more of an expression to get his (passionate) point across. It’s not meant to be like “YOU ARE LYING, DOUCHE!”
Neito: You’ve got a solid philosophy or researching a reviewer and his or her tastes by latching on to similarities. That’s a rad way to do it. That’s how I discovered that I basically trust every single person at OXM. Well, except for that time Casey gave Turning Point a 6.5 instead of the 1.0 it (seriously) deserved… Just messin’ Case.
Oleetku: So it sounds like you’re looking for the personalization of a lot of Future magazines, or a 1up.com, with the broad-stroke explanation of how a game works like GameSpot.com does. Sounds like you’d enjoy Francesca Reyes, or Rob Smith, without a word count.
I’m surprised that, despite my sarcasm, the money hat thing still comes up, and none of the discussion brings up any evidence beyond “I think.”
Fuzzy: If you don’t read someone’s review, you’ll never know their taste! You make the same point about films. Have you tried following a reviewer and it just failed? A magazine, perhaps?
Arrogance: Re – your first paragraph: duly noted. You make good points, but it sounds like you’re only enjoying a game for its specific things. If they’re different from before, that’s usually a good thing. I liked ALL of that stuff in Burnout, hence, I agreed with the reviews. Same for Soul Calibur. What you’re asking for is that minutiae be spelled out letter by letter. Well, IGN does that, because they seemingly have NO letter count (I joked about GameSpot not having one earlier, but they do), but people have taken issue with that for various reasons.
Outside of that, I think you make a critical point: “Really, if we aren’t playing for ourselves anyway, who are we playing for?” – It really does boil down to that. I like FPSs, but Far Cry 2 looks absolutely boring and generic to me, yet it received 8s, 9s and 10s across the board. It is an extremely personal thing, and one thing I despise about being a reviewer is that I CAN NOT appeal to everyone who’s reading my stuff. And I don’t know WHO I’m appealing to.
Ezilylost13: I’m with you there. Integrity of a reviewer is as subjective as their review (unless they’ve done something baaaad). I know a freelancer, and I’m absolutely fucking IN LOVE with his work. Sometimes our opinions differ (Far Cry 2, CoD5) but generally I trust him. The guy has come under fire a couple of times, but I still think he’s got a solid way of seeing things.
Same goes for big outlets. I’ve seen cover stories in OXM get lower scores, GameSpot give out low scores to games they’ve been advertising (Spiderman, Kane and Lynch), and other stuff like that. The integrity isn’t nearly as crucial to me as it is to who I’m reading, but stuff like that certainly helps.
Goffey: ”Site X is wrong because I would totally give game Y a 9 not a 8″ Tom Chick brought up a point that I really latched on to during Shawn Elliot’s symposium, which I’ve now linked in the original article (I forgot! Yikes!) — If someone reads a review and says “I would have 9′d it,” that is a conversation that’ll never happen. He thinks that if you were to eliminate numbers, you could have thoughtful discussions about why a reader “didn’t like the story as much as you did” or “thought the combat was better than how you made it sound.” That’s a really excellent point — aside from scrapping the scores, and all that. I think scores are WHY people read the review. The text is a tertiary part of it.
And that’s heartbreaking when you’ve got 1200 words explaining why you gave it a 5.0.
@Mitchell Dyer
Yeah I don’t think Dan was being condescending at all man.His responses were spot on.
Arrogance: You can actually cancel a race in Burnout Paradise. Just stop the car and wait a few seconds. I could go into a whole discussion over whether that game was progress or not for the series, but that’s for a different time and place.
Otherwise I think it almost makes the question of are you looking for a review, or some other kind of breakdown of a game. As far as I’m concerned, the “Shit or Brilliant” system is a complete way to review something. It’s giving some sort of rating to something based on some amount of “goodness” there in. How “goodness” is decided, ratio of shit to brilliance, particular details of it are all left to the wayside, but it remains. It has been reviewed and judged. Some people might actually be looking for something like this. The kind of people who go after number ratings. But then you look at something like the Zero Punctuation reviews, and they are in general good at going over good and bad parts of games and giving some specific ideas of the experience, but some people come away from it asking “well, was it worth playing or not?”
I guess I feel that there’s some sort of difference between reviewing, and deconstructing/explaining a game. If there is a difference, then is the question “What should magazines the magazine’s be printing?” and “Can we have both?”
It is very rare that I ever read a review. I feel like there are so many outlets now where games are discussed and dissected, often before the reviews even come out, that majority of reviews would just give the bare bones rundown about what I’ve either already heard about on a podcast, or can actively discuss with other people in real life or on forums.
I do love to read articles about video games, though. As I’m trying to organize my thoughts here, I’m realizing that I have, in my mind, separated game reviews from game criticism, even though a review is nothing else but a critical look at a game. I think that the things websites and magazines typically throw under the “Reviews” header aren’t enough. Games are deeply complicated things to talk about these days. Rarely does it feel writers are given the word count to hit all the bullet points readers, for better or worse, are looking to hear about, and the solution to that we sometimes get are just numeric breakdowns of Story, Gameplay, Graphics and Sound that average together into a number that doesn’t mean anything to me.
I’m just naturally inclined to be dismissive of reviews these days, even though a lot of talented people I really respect are working to turn it around. I worship the work of the 1Up team, but the absolute last thing I checked out on that site, after tearing through all the podcasts and feature articles and editor blogs, were the reviews, written by the same people who I loved to hear on 1Up Yours or GFW Radio. And their reviews were great. But ultimately, they were still working on a somewhat cramped canvas. I’d rather hear Shane talk about a game for 10 minutes than spend a minute reading his review. That’s probably the opposite of what most people want. . . the web is a place to get in, get your info, and get out. But for me, there are just more fulfilling ways people are expressing their thoughts on a game than the average game review.
What would make a game review fulfilling, to me, would be to let reviewers just write what they felt, within reason, with their own voice. I felt the reviews on 1Up had personality, and didn’t just tell me that the story was a 9 but the graphics were a 5, so the game overall is a 7. When was the last time you read a professional movie review that rated the story, the acting and cinematography separate and averaged those scores together? Or even better yet, a book review where the story, the characters and the themes were all treated like separate parts of a whole? It’s the difference between reviewing hardware and software. When I look up a review of a television, yes, I want the bullet points. How’s the picture? How’s the sound? What connections does it support? When I want a game, I want to know what my experience will be. A game isn’t a soulless machine. . . they are, dare I say, art. Well, maybe not art, but they are a creative product, and deserve to be reviewed in creative ways.
I do read reviews pretty regularly. With money being so tight, I like to have a few different points of view about a game that I’m considering laying the cash down for before I make that crucial decision. Unfortunately, I can no longer afford to be the impulse buyer that I was even just a couple of years ago.
All facets of the review are important to me, including scoring. While a part of me would like to see scoring go by the wayside so that more people will actually *read* what the reviewer takes time to write about a game, another part of me does appreciate scoring and its ability to make reviews easily accessible for readers who don’t have immediate time to read 1,000+ word pieces.
In terms of scoring, though, I’ve always been a proponent of non-numeric scoring… like a Skip it, Rent it, Buy it tiered system with an occasional special award if the editorial staff all agree that the game is truly remarkable. Numeric systems, in my opinion, usually aren’t scaled right.
Lastly, I don’t think that review writers can be “bought”… unless they allow themselves to be. It doesn’t matter to me if you send me a review copy of Super Bombad Racing and a bunch of Star Wars swag– the game is still crap and it’s my responsibility to let my readership know this, along with supporting evidence as to why I think so.
(P.S. I’m loving the new blog, Mitch.)
Anotehr thing to look at is why do we need a numerical score for games?
Peoples tastes in games are so widespread that the difference between a 7 and a 9 could be so miniscule.
Take movie ratings for example. A critic could give a movie 3 out of 5 stars and say “it’s not the best movie but worth watching”…ummm ok. So basically watch the movie yourself and then form your own opinion on it, but if a 3 is still telling you to go see the movie, is there really a point to having the 4 and 5 stars there?
Personally I think the best way to “review” a game would be similar….as in Buy, rent, avoid…as long as the review explains why, to them, that the game is or isn’t worth playing.
Peter: yeah, exactly(I was writting my post before i got a chance to read yours)
MCBurnett: I’m with you on the podcast thing. The stream of consciousness and relation of experience is something I love to hear about, and conveying that in a review is hard because it’s so much more difficult to hold someone’s attention in text. Telling a Left 4 Dead story, or how fucking crazy it was when you caused a crazy amount of chaos in GTA IV, is much tougher to engage someone in through text.
I’m not saying you can’t, because you certainly can, but it’s easier to listen to ANYONE ramble about an awesome moment they experienced that makes them like, and continue to play, the game in question. With a review, you don’t get tone or expression, so it’s harder to gauge how enthusiastic someone is, which parts they liked specifically, and all that kind of stuff. I definitely love reviews, but I think if I could listen to a 1upYours (RIP) segment on Prince of Persia, or read a review, I’d opt to listen to back-and-forth discussion than someone, who I may or may not agree with, tell me what works and what doesn’t.
Again, both are totally viable and great, but I think that podcasts are a fantastic alternative.
Peter: Thanks
Goffey: (this crossover posting we’re getting is unreal) – I’m kind of with you on numbers. If you’ve got context as to an outlets scale, it’s fine. OXM has 50 word descriptions as to what a 7-7.5 means, and why it’s different from an 8.0 or 6.5. Places like IGN use a 100 point scale, while Game Informer uses this weird middle-ground with .25 point increments. That kind of stuff makes me wonder.
When you give Mass Effect a 9.75, it might as well be a 10. When the quarter is docked off because the critic says the combat is terrible, like GI did, it’s ultra-confusing.
If GTA had received a 9.9 or 9.8 from IGN, would anybody care? I think the only complaints would be, “well, it’s already there, just round it up the .1 or .2″ When you get to that stuff, you’ve got math getting involved, where you’ve got this segmented batch of 9.6s and 5.3s to attach to graphics and gameplay.
IGN doesn’t average their ratings, but with numbers like that, it’s just confusing.
It’s almost a wonder as to why have the lower numbers. not even just in the sense of “no one uses them anyway” but also in the sense of just how horrible does something have to be for you to not even mention it? Is it that you’re going to try and rate everything that comes along? Are you taking into account that at least some amount of work has to go into it to make it a commercial release? What would constitute a 1 on a scale of, let’s say, 1-10?
I think lower numbers are fully warranted. Sure, you might totally brush off a game that gets anything less than a 5, but it’s worth knowing what it does wrong. Maybe an atrocious game has an amazing story, and that’s worth addressing not only in the text, but the score as well.
And what constitutes a 1.0 is Turning Point: Fall of Liberty. That is literally the worst game I’ve ever played. And that’s not just me being a baby.
I think there is no way to totally lose the number scale as NA is facinated with numbers. Higher numbers equals success in NA. We want instant gratification and an instant answer to a question.
Everything is based on numbers….so and so is good because it’s increased X%, this game is more of a success because it’s sold a higher number of copies.
Look at the sport of Soccer in NA. Sure it’s the most popular sport in the world but the majority of the people following it are from Europe or South America. it’s mainly because a soccer score doesn’t flat out tell you how the game was played out….ask an American why they don’t watch soccer…the likely reply will be “it’s not exciting enough and the scoring is too low”. Really? well if you ask someone who understands the sport and the intricacies that go along with it they could tell you that even if a score of a match is 1-0, the team that won could have flat out dominated the team that did not score.
Also we north Americans like to use numbers to back up supposed “facts” because well, we’re lazy. It’s easy to say your favourite game is better than someone else’s simply by saying “well my fav game got a 9 on gamerankings, you’re only has a 7.5″ Imagine the insanity that would take place if people were forced to back up their opinions by actually disecting a game themselves in a discussion if the rating system was as simple as the one I mentioned above…..”Yeah well my game got a buy”….”yeah well so did mine”….”ummm ok…then WHY do you feel that game is overall a better choise?” IT’s MADNESS!!!
People(especially in NA) want instant gratification and unfortunately we’re trained to judge success and greatness by a numerical value.
You’re really on with the gratification thing. People want to believe that something they’re interested in is worth their cash, or that their purchase wasn’t in vain.
Well, Here I am reading through this. And figured I would offer up my 2 cents, (still mad that they don’t have the cents sign on my computer)
Anywho, It’s been quite a while since I’ve sat down and read a review for a game. Though 100% of this is because I have yet to buy a next gen console. Only recently have I been buying games again, and they’re all for my PC. The games I did buy I still haven’t read any reviews for, I just went to the best source I had, Mitch. I used to read OPM religiously, I got to know which reviewers I shared opinions with, who to trust on what genre of game, but as PS3 came around I stopped buying the now defunct magazine. I had no interest in reading reviews on games I wasn’t going to buy. When I started PC gaming, the only persons opinion I could trust is Mitch’s, but we’ve been playing pretty much everything together since we rented Turtles In Time from Video Vision. He knows what I like, I trust his judgment in pretty much all things game related.
Now this isn’t me saying I’m only going to read Mitch’s reviews. I’m saying I haven’t been able to develop any opinion on any other reviewers, I have yet to learn who I agree and disagree with on any given subject, and that’s the most important thing about a review. I know it’s been said before, but a review is ultimately one persons opinion, all we can do is try and find someone who’s opinion we generally agree with. I’ll read anybody’s review if I’ve got any interest in the game, But if I don’t know the person I won’t have any idea how to feel about their review until after I’ve played the game myself.
Now onto the issue of People being Bought Off. I honestly don’t see why this would happen in the world of games and game reviews. I know it always looks good on the back of a game to say “4.5/5″, but what I would like to believe to be a majority of gamers, don’t base their purchases on what was written on the back of the box. In this day and age where everybody can make their voices heard, if a game was given a review by a single person that it did not deserve, that single reviewers voice will be muffled out by the virtual words of the consumer. I also don’t feel that a reviewer would put their credibility on the line for a few extra bucks. If a respectable reviewer gave Star Wars Super Bombad Racing a 9/10, I’m not reading anything they have to say again. I’m sure the reviewers have considered this as well, which is why I think that we don’t need to worry about buy offs.
Just my opinions folks. I’m not a very trusting person, I need to know who’s reviewing the game before I will truly listen to what they have to say.
I have never NEVER used game reviews as my sole reason to buy a game. I dont care if the game 10 out of 10’s on every gaming publication, unless the game looks appealing to me, I see no reason for me to trust the reviewers opinion because there not me. For example, I dont like real time strategy games, so if an rts were to hit the shelves and get great reviews across the board I know that im still not gonna pick it up because I dont like that paticular genre in the least. People need to remember that a reviews are opinion not fact.
I try to read reviews for everything, videogames are something that especially interest me. Unfortunately most game reviews are terrible. They’re systematic, made on a standard review template that chops up whats great about the game. Games For Windows magazine had the best reviews, and when I find some systematic shit I just stop right away. Usually terms like “crisp audio”, “slick visuals”, “compelling gameplay”, “intuitive design/controls”, and the like are an instant turn off.
I’ve really only bought 1 game based on reviews alone. That game was the original God of War.
I sold my PS2 months before it cam out and was primarily gaming on an original Xbox, so I wasn’t really following anything outside of what was comming to Xbox.
Once the reviews started coming in for that game it peaked my interest. I ended up buying a brand new PS2 slim JUST for a game that I had no idea whether or not I’d like.
GoW is now one of my favourite game franchises of all time.
Goffey: Spot on. A review is subjective and one person’s opinion based on experience. You take it or leave it based on your own preferences. It’s not a law, and it should not threaten anyone else’s independent conclusions. The fact that people do react as if it’s a threat is not a problem with reviews, but in people’s interpretation of them.
Mitchy: “The last time I even faced an ethical dilemma like that was in — no joke — 1995. And I took my name off the review after it was sweetened (from a 2 out of 10 to a 5 out of 10…was that really even worth it?).”
If you can, could you possibly elaborate on why a score would be juiced like that? I mean, it’s easy to assume that there’s a bag of money if you sign the papers, so I’d like to be a bit clear on that, if we could.”
Yeah, it was juiced out of fear of losing favor with an advertiser. The publisher of that magazine said “C’mon, it can’t be that bad…” and I said “No, it really is.” And he was thinking, they’re going to pull advertising if we dump on their Genesis game. So it was very much money motivated, but the publisher was thinking “I don’t want to lose money” and not about “I am being paid to deliver good scores.”
That kind of “are you suuuuuure the game is that bad?” happens frequently — I mean, every reader asks that question too, especially if they were expecting a higher verdict. The money guys ask the same question when they want to take the temperature — but they don’t dictate the weather. If it’s bad news, they want to be prepared when they go talk to that client (and, as Shoe has explained in the past, publishers do threaten and actually do pull ads in a snit if they aren’t happy with scores). But, at least at Future, our moneyhats don’t say “no, it has to be higher, change it.” That’s simply never happened to me here.
So that one time (and I realize now it was 1994, when that game came out) comes to mind, several magazines ago, and it was simply, “we cannot print this score because it’s too low, change it.” It was out of financial fear, not mythical bribery.
The magazine and the company that made that game no longer exist. The publisher is still around but does not work in the gaming sector, and was otherwise a great guy.
And the funny thing? The user reviews on this game today, on sites like Giant Bomb and GameSpot, are very high. They make it sound like it was a pinnacle of Genesis gaming, and my — subjective — experience was very different.
I’m dying to know what this game is you speak of Dan
Combat Cars, for Genesis, from Accolade. Top-down 2D racer with no map options.
(And yeah, dj, I didn’t mean to be condescending, glad you didn’t take offense — I come on strong sometimes in how I state my views, but I really did not mean any personal offense.)
I might be a bit biased considering I also work on a gaming site. I honestly would prefer not to use scoring (regardless of points, stars or letters) because numbers don’t mean anything. We might say ‘that girl is a 10′ but what does that mean exactly?
The problem is that we do not have the time to go through long and sometimes drawn out reviews to get all the information we want. We use scores because we can simply associate a number and determine a price. If one game is a 9 and another is a 7, people automatically assume the 9 is better and will be more likely to purchase not bothering to reading the review of the 7. Sad part is that while the game might be scored less it might be more to their tastes.
While I would go away with the Scores, I do like that many sites are going the way of giving you Pros and Cons for each game. The ‘plot summary’ of a review gives a person more information in regards to a game/movie/tv etc… and can help them determine if the purchase is necessary. I see that being used more often since it gets the underlying message across. If a person wants more detail in terms of what the game entails, then a small blurb to coincide can do the job.
Sonic the Hedgehog? That shit is worth the two.
CuppuhT’s sentiments sound about right to me. Knowing the guy might help, but his arguments hit on a lot of interesting spots.
So it’s not Sonic!
I’ve never heard of Combat Cars.
Combat Cars eh?….don’t recall that one and from the sound of things, that’s likely a good thing lol.
@MarkoDj: I agree with what you said but one thing stands out
“The problem is that we do not have the time to go through long and sometimes drawn out reviews to get all the information we want. We use scores because we can simply associate a number and determine a price”
That statement right there tells me that people want to get the most information they can but don’t want to put the effort into reading it, therefore people use a number to substitute valuable info.
Or am I taking that the wrong way?
If not, I totally agree. People do want in depth reviews but at the same time some people want the quick fix. There will likely be no happy medium which is why there will always be gamers calling out sites, mags and other gaming media since they will either disagree with the score given or the text that goes with it.
I personally love when mags or sites use the pros and cons blurb(like Dan and the rest of OXM does with reviews) the blurbs coupled with the 3 tier rating system to me would be perfect(IMO)
A score to me is simply more for bragging rights.
River: What do you suggest in place of those descriptions? Sure, you elaborate on why the controls work well or not in your review, but do you have a specific reference or idea of something that you’d like to see instead?
And I agree with the GFW sentiment. Their reviews truly were criticism. Really fantastic reviews on that end. I still to back and read 1up PC reviews for inspiration.
I usually ignore reviews, personally. At least, I ignore “proper” reviews like IGN and the various magazines. What i do pay attention to is what people like Penny Arcade and Fast Karate say. I had never heard of The World Ends With You, because I tend to ignore gaming news, but after reading about it in the PA newsposts, I was intrigued enough to purchase it. And I loved it. If I owned a PS3, i’d buy Valkyria Chronicles because of what Dave has said about it. FK talked up Portal so much that I bought the Orange Box, and immediately fell in love. Portal is now in my top 3 games in the history of anything ever.
Other than that, I just talk to my friends who have played the games, and if possible I play their copies for a bit. I’m friends with these people because we usually have similar tastes. I own Bioshock, Left 4 Dead, and Mass Effect because of my friend Badwick raving about them, and I plan to get CoD4 for similar reasons. The only major issue with this system is that I tend to be behind the curve when playing games, which is why I’m only just now getting into Left 4 Dead and Bioshock.
Of course, I also tend to have some other opinions of games, based on my own tastes and what I’ve heard from others. For example, I don’t have any interest in Fable 2 or Kane and Lynch, despite the fact that my fiends are in love with those games. Same with Metal Gear Solid. i actually feel a bit bad about it, to tell the truth, but I’ve never liked MGS or it’s sequels, despite them being legendary games. So no matter what the reviews or my friends say, certain games I’m never going to buy.
Having said all that, I do pay attention to reviewers that entertain me. I’ll always watch a Yahtzee review, because he makes me giggle like a madman, but I tend to ignore his opinion of games, watching more for the humor than anything else. I can’t even remember the last time i looked at a “legitimate” review, or ever cared about the number assigned to a game. My “buy” decisions come from more of a mental osmosis process. It might not be ENTIRELY accurate, but so far I’ve never bought a game that I don’t like, so the system hasn’t failed me yet.
Mitch:
I’d say that game analysis is something that people are going about all wrong. Obviously a game is a culmination of many different parts, what you see, what you play. What you hear, who you hear, etc. etc.
Atomizing the game and essentially ‘taking it apart’ is important. Why is the game evoking such an emotional reaction in you (whatever emotion that may be)? Put the game back together and see what’s doing things right and what’s doing things wrong. What is ruining the experience, what is contributing to wholly immersing you or delivering on an experience? Not what you like, but why do you like it? What’s making you like it?
Saying Gears of War 2 has great graphics doesn’t help me, I can see screenshots and judge that for myself. Great graphics is also something subjective, so well you may think Gears has great graphics, I prefer an environment with more colour that I can pump up the settings to, i.e. Crysis on my gaming rig. Or the people who would dismiss Gears’ art style instantly and gravitate toward Banjo Kazooie, or something.
Its also unnecessary to praise a gaming on something like the examples I gave. None of them have any meaning. Seriously, what the fuck does slick visuals even mean? Is the gameplay really compelling? Do you even know what compelling means? So lets say the gameplay is truly compelling, so what? That doesn’t make the game good, and this is the fundamental issue with AAA games constantly getting much better scores PURELY on their production value. Really? Bungie has a lot of money to hire celebrity voice actors? Great, the voice acting is good. Is what they say interesting? Is it deeper than the standard game story’s “RECOUNTING OF EVENTS, SOME MISSIONS, IF YOU’RE LUCKY, A TWIST.”
I keep going on random digresses but I really like the way its going. So I’ll continue with another digression.
I praised GFW but I don’t mean for that to take away from most other publications. I love OXM too, I think they’re far ahead much of the other trash, and the trash is Gamepro/Game Informer. IGN is a step above them, but they still don’t give a shit about games any more than doing their “this, but this, this but this, this but this, conclusion” reviews.
Shawn Elliott (one of the former staffers at GFW magazine and 1UP. doing the excellent review symposium you mentioned) said at one point, what if by making a review you essentially had a thesis to write? Like, you make a thesis then go about proving it.
There’s also the failure in granularity of the score, whatever you do a 9 game is going to get compared to a 7 game even if the reviewers use different scales. There’s too much granularity even in a 10 point scale. Giant Bomb’s 5 point scale is closer to the mark, for me. But the score means essentially nothing, and I’ve written reviews before (never for pay, unfortunately) and I know when it comes to choosing the number there is no objective formula applied to the game, and there is no real connection between when you finish the review and decide on a number. Yet the number is what everyone wants to whine about on the internet. Its ridiculous.
ABOVE ALL,
Reviews need to change to begin a discussion on games, not act as though they are the end-all be-all, final word on a game. Seeing OXM give Dead Space a 6/10 and seeing it get a 5/5 stars from Giant Bomb should say enough that people have differing opinions.
A review should have you say to yourself “oh, I never realized THAT’S why I thought it was awesome!” Or something of those lines. It should point out things to you that enrich your experience with the game. I’m a huge Valve fan, Half Life being my favourite game series, and I always listen to all the Developer Commentary they put in their games. Its incredibly insightful, extremely interesting, and just all around entertaining. You see some of the design choices they made and there’s so much care put into them. Just in Left 4 Dead there was a commentary about how they put dark buildings on a brighter background (in the backdrop skybox of the map) because it allows the buildings to look much less rectangular, and therefore more natural. I never consciously noticed that, but that would be a great point to bring up when talking about how Left 4 Dead is extremely immersive in the way it pulls you into a zombie apocalypse, and the way it very rarely shows its week point, like much of the Half Life games.
@Goffey
You understood me correctly. I made a mistake with the last word ‘price’ as I meant to say ‘determine a purchase.’
I do agree that the score is more for PR reasons than for gamers. I think gamers care how the game plays while the companies want a score so they can say ‘look at us, we’re loved by all.’ Even in situations where a game is reviewed poorly, a PR firm can still sift through reviews and find positives which will spin their game.
In the end, I believe that reviews can’t be too short nor should they be too long. If you try to give someone information in 100-200 words, chances are, you’re only giving them only a fraction of the information. But at the same time, most people don’t want to have to go through a novel in order to get an idea of a game. That is why Previews exist; give people some information and then they can determine if they’ll a) continue to track the game’s progression before it hits store shelves of b) pass on it and find another game they might enjoy.
—-
On another point which I missed earlier was the talk about ‘Reviewers being bought’. I think this is false to some degree. Freelance reviewers (like myself) are rarely the focus of gaming companies. I have only done freelancing for about 4 months and while I’ve done quite a few reviews, I’ve only been contacted once by a company and that was after my review was posted. They tried to state that there was an error in my review, but I managed to prove them incorrectly (it had to do with a feature in the game; they said my information was false, but then I just had to quote the last page of the instruction manual and they became quite silent).
While Freelance reviews in my opinion are free of direct influence, I am worried that they/we might be inclined to boost a game in order to look good in front of the the site(s) we’re working for or for future work we may be in need of. I can truthfully state I have never done that in my short time, but I do sometimes wonder if others, especially younger writers, looking to get a real foot in the door, are willing to make publishers happy so that they’re remembered in a positive light in the future. Who wants to be known as the evil reviewer (we all remember The Simpsons Episode where Homer is a Food Critic)
Lastly, I do sometimes fear that while reviewers aren’t influenced, I think sometimes copy editors are, especially if they are aware of the game prior to the review. If the Copy Editor (or the Editor who proof reads prior to publication) doesn’t like a certain aspect of a review, they can simply remove portions and just say ‘oh it was irrelevant to your review’. That is something that worries me because they do have more direct contact with the publishers and they know that if the gaming companies stop supplying them with games to write about, the site suffers and they’ll be out of a job.
Again, I’m new in the game. My lack of experience puts me in a situation where I am spending more time learning by reading blogs like this (fantastic by the way Mitch) and listening to game related Podcasts.
To all of Dan’s excellent points which I will not quote because scrolling up is old and busted:
I think in general you’re right to argue vigorously against what “everyone knows” about gaming journalism and who pays for what. But generally accusations towards specific reviewers tend to be an emotional rejection of a particular score or a general disagreement with the reviewers overall tastes. Good or bad, opinions are pulled out of your ass, and sometimes people forget to pull their heads out before they start talking.
This raises the side question of how to respond in a way that defines your integrity for your readership. Obviously you’re unlikely to change the mind of the person attacking, but usually they do so in a public comment, which leaves you with the burden of response, as well as (and this is impossibly unfair) the burden of proof. Really, the person attacking should have to prove their case, but when other people are watching the confrontation you really don’t have the luxury of waiting for the attacker to go away if you want to retain your readership.
So side question to the reviewers who are participating: have you faced such personal accusations before? How did you respond? How’d that work for you?
I think the more insidious accusation however is that of institutional bias. The official mags face this nearly constantly, but pretty much every publication has come under fire at some point or another.
(Give it about two months and I guarantee that someone will accuse Yahtzee of getting paid off by Sega to review Sonic badly to cause a reverse-psychological effect that makes people want to buy it.)
The problem is that gaming publication receive advertising dollars from publishers. Consumers are very often smugly couched in their cynicism, confident that they know what’s really up and OF COURSE all reviews are bogus and paid for because they love their advertising-bought sports cars. It’s the same mentality that makes most people secretly suspect one conspiracy or another: conspiracy theories only prove to the believer that they are smarter than everyone else. It’s just so damned gratifying.
So the belief in institutionalized corruption is much more difficult to eradicate. People really want to believe that it’s there, and it’s difficult to argue from a position of strength when your company does in fact take money from game companies. It’s difficult for the outsider to accept that no corruption occurs when the temptation is so baldly there.
The elephant in the room, though, is that IF something shady occurs, this is where it would happen.
Transparency will help any publication here, I think. Most can’t afford the luxury of only advertising things they like (a la Penny-Arcade), unless their business is built from the ground with that point in mind (with therefore different forecasts for advertising revenues). An explanation of how reviewers are chosen for given games, as well as general data on a reviewer’s tastes would help as well. Nothing that needs to be in every print copy, but maybe something extensive on the website to direct detractors to.
Meanwhile, publishers can do more by making it a very public (and real) policy to give or pull advertising dollars based solely on readership and demographics data, not review scores. This is the only model that makes sense anyway, because it’s in their best interests for publications to appear as incorruptible as possible. Just like Dan isn’t willing to lose his job for a little side bonus, publishers shouldn’t be willing to lose advertising audience for a bump from 7.8 to 7.9.
This isn’t going away otherwise, and its one of the core things that’s poisoning the entire industry. The perception of crookedness.
Review scores:
A couple of people have brought up that the scoring systems are inconsistent, and I’d like to expand on that. More problematic than differences between publications are the inconsistent gradients that exist in most scales. There’s a lot of nuance that tends to go into the top-tier scores (7-10), whereas lower scores tend to be applied as a rubber-stamp of suck.
Some of that’s human nature, I suppose, because we’ll take perfection over flaws any day, but I think there’s a sort of psychological pressure to not review a game below 7 or so unless it’s REALLY bad.
I’m not saying anything enforced, in fact I suspect it’s largely subconscious; something inherent to the industry. The problem is simply that I guarantee that as much as I hate a game, you probably love it.
Case in point: There was an energetic discussion about Beyond Good and Evil over at Fast Karate recently, wherein myself, Mitch, and others were praising the game, while Dave Riley and others blasted it. There were plenty of valid points on both sides.
Dave and I never agree (except about BGC 2040)We debated for a week or more once on whether Metroid Prime 2 was better than MP1, or if instead it was a flying turdburger. We agreed to disagree, and promised to not send ninjas to hunt each other down.
Point being, as bad as he thought the game was, I loved it. A reviewer who hates a game is usually well aware that others will find things to like, and so as often as not it seems like grades of 6 or below tend to be more a measure of whether there’s something there which others might like and not the game as a whole, which seems sort of contrary. I think review scores tend to encourage reviewers to review not as themselves, but more as some fictional everyman who both loves and hates all games for all the valid reasons in the world.
An interesting side effect of this is that readers tend to create their own internal calculus which takes a review score and translates it into what their own experience would be. Exactly the way we do with windchill figures. This internal figuring is I think the primary reason review scores persist; they aren’t a perfect solution, but they’re at least partially effective at translating my experience based on my tastes into your own expected experience.
I don’t like them because I think they subtly change the way reviewers write because they’ve got to think about the theoretical experiences of other people, not just themselves. But I think they’re necessary.
With any system, it will only go away when something better comes along, so my challenge to review score detractors (who have many valid points) is to devise a system which provides that crucial communication which crosses the boundaries of personal tastes, and yet is less intrusive in the actual critical process.
In the meantime, publications could get some mileage from allowing users to turn review scores on and off for their websites.
That’s a lot of words. Time to go watch the game. I’m late!
“There’s also the failure in granularity of the score, whatever you do a 9 game is going to get compared to a 7 game even if the reviewers use different scales. There’s too much granularity even in a 10 point scale.”
Thinking about it a little, I’m actually starting to think that the number system isn’t too bad, but only in the situation of a person following a single reviewer who they share opinions and values with. Because then, nothing’s colliding, and all you need is a number, because the relationship basically becomes the reviewer saying “I’ve gone through all these games, and these are the ones that are importaint to us.” It’s almost like the reviewer is a secretary, telling you what games should be on your schedule so you can make the final decision for yourself.
Of course, if you don’t entirely trust the reviewer and you need to compare between reviewers before you make your decision, then the number system breaks down and only the word matter.
River said:
“Saying Gears of War 2 has great graphics doesn’t help me, I can see screenshots and judge that for myself.”
Graphics are really important to some people, though its incomprehensible to me. I know a guy who couldn’t believe I disliked Dead Space, but when he found out I played it on a regular TV, not HD, he latched onto that immediately as the reason why my enjoyment didn’t match his. Totally missed my complaints about a lack of pacing or strong characterization. But that’s important to him.
Not me obviously, as I have a regular TV still.
@ Matthew
But we are in agreement that this is a huge fallacy, right? Its pretty much idiotic to think incredible graphics has THAT much to do with your experience, and the effect the art style has is MUCH GREATER and much more important.
I want to make reviews better, not make them for the general idiotic populace who things 7-8 is the average between 1 and 10.
Holy crap, this thing has exploded .I went to beds and there were like 15 posts or so, and when I woke up there were 50.
Don’t you think the numbers system is kinda broken though? I mean, some poor reviewer has to write a 5 page article and then all the gamers scrutinize is the final 68/100 score?
Has anyone tried perhaps ending the review with a summed up 200 word paragraph, giving his thought on the game and forego the scoring system? (Sorry if someone has done that, and I’m an ignorant fool)
@ River
I’m actually not in agreement that this is a huge fallacy, despite my own personal disinterest in quality of graphics. I think they’ve been blown out of proportion for a long time now, and I think we’re entering the murky realm of the uncanny valley, but I can’t develop an argument which presents a preference for high-end graphics as fallacious. The problem is that we’re discussing areas of perception, attributes of a program which provides stimuli to the senses, and I can’t make a blanket statement that says, for example, artistic visuals > photorealism.
Photo-realism is itself a style of art, and I would argue two things in its defense. One, the artistic style of an Okami or a Braid would be absurd in a story about war and brutality (however badly written) like Gears 2. Two, your reaction to it is enthusiastic or lukewarm in direct proportion to how much escapism you’re looking for. I have a friend, for example, who will not read anything but high fantasy or science fiction, not because other genre’s aren’t as good, but because he believes that if he wants an experience close to life, he will simply live, and not bother with the book. If he’s transporting himself to a different world, he doesn’t want it to be much like this one.
The appreciation of photo-realistic graphics encounters the same divergence of what people are looking for. For some people photo-realism does nothing because they can simply look to the left or right and see the same level of detailing. For others its appreciated because its used to detail a situation not found in their normal world. Still just difference of tastes.
This is the essential balancing act which reviews have to try to address. My Dead Space example is a little extreme, but not that much. How do you take a very focused opinion and market it for a mass audience? You skew average to the upper end a bit, and hope someone comes up with a better system of communicating focused opinion to a diverse audience.
Here’s something that Dan Amrich wrote that WordPress/I accidentally deleted earlier today.
“I really like the three-tier rating system — the “buy it, rent it, skip it” concept is, to me, really pure. If you asked a friend “what did you think of this game?” they would say one of those things — “it’s great, you should get it,” or “eh, it’s okay,” or “dude, don’t bother.” I don’t have a problem with the media using the friend scale, and I like the publications that do.
I did exactly this today, Dan. Cuppuh-T from above expressed interest in Resistance for PS3, which I quickly shot down with “NO! Don’t get it. It’s way generic.” and that was all he needed. As mentioned, we’ve known each other for YEARS, so that was all it took to convince him that he didn’t need/want the game. Various Steam recommendations have lead to purchases and cart removals on both our ends, with simple “Ah, that’s not so hot” or “Yeah it’s badass, buy that” comments.
Crispy Gamer is doing something like that with their Buy/Try/Fry ratings. I don’t like the expression “Try it” because it implies that anyone can rent/borrow with ease. It’s simply not the case, and I almost wish they reworded it. Problem is, I’ve got no alternative suggestions.
Looking back, I think my last port is kind of shallow and doesn’t quite address the topic. So I’m gonna throw in another post.
One thing that i think reviews lack is any mention of the universe of a game. Reviewers us words like “immersive” (not a word according to FireFox spellcheck) or “detailed” to describe a game universe, and leave it at that. When I read a couple reviews of Mass Effect, for example, reviewers mentioned “it’s a rich, detailed universe” but I never got any feel from them just HOW deep it was. But, listening to Dave Riley talk about it on Fast Karate, I got a much better sense of the universe, and yet all he did was babble some semi-nonsense about space Jews. Yes, the combat’s kind of wonky, the game’s a bit glitchy and laggy, and some of the story scenes are delivered awkwardly, but the universe was so wonderfully detailed that I fell in love with the game, and now it’s easily one of my top ten game experiences. Yet the reviews don’t seem to focus on that.
It may just be my personal preference, but still, I’d like reviewers to give more focus to the universe- when it’s applicable, of course. A game like Need For Speed doesn’t need universe-building, but a game like Halo does. I primarily don’t like Halo because it has a shoddy universe. The first game didn’t give me enough of a sense of who I was or what I was really doing for me to care about it. Meanwhile, in Half-Life, I have a very clear understanding of who I am and what I’m doing. Purely gameplay-wise, I think Halo is stronger than Half-Life. Yet, I near-obsessively love Half-Life, while I have almost no interest whatsoever in Halo. It’s the universe, or maybe just the storytelling, that draws me in.
What I think reviews need is a way to effectively deliver a strong impression of gameplay, technical aspects, and storytelling/universe building.
On a slightly related note, whenever a certain reviewer catches my attention, I look at the archives and figure out their opinions of games I’ve already played, to help me figure out how much weight to give their opinion. So far, the system works, as I don’t actually own a game that i don’t like (except Virtua Fighter 5, but that was all I was allowed to buy when I first got my 360. Little sisters are the bane of my gaming life.)
Hopefully that addresses the conversation a bit better than my last rambling post.
I latch on to podcasts and other places for recommendations and opinions as well. Dave Riley has similar taste to me, but he has the odd game that either sticks out as amazing to him (which I totally enjoy but don’t go bananas over) or something he hates that I enjoy (Call of Duty single-player, Metal Gear Solid 1-3).
Regardless of his taste, he is always able to articulate why he does or doesn’t like something, and hearing him speak or reading him write about a game is always enough for me to go “Alright, I can get behind this game/ Okee doke, maybe this isn’t for me.”
IGN has one person who I basically “trust” without needing to consider anything. Greg Miller.
OXM’s Paul Curthoys and I align eerily close in like/dislike for genres/games. For that matter, I pretty much trust that entire staff.
@River: “Reviews need to change to begin a discussion on games, not act as though they are the end-all be-all, final word on a game. Seeing OXM give Dead Space a 6/10 and seeing it get a 5/5 stars from Giant Bomb should say enough that people have differing opinions.”
I would argue that reviews don’t need to change, but the audience needs to change in how they perceive them, for exactly the point you cite. When I write a review, I do not intend a review I write to be the end-all, be-all, final word…but I do have to say something authoritative that I can defend and justify, or else it shouldn’t be printed at all. The idea is that the person writing that review is an authority on the subject, so the text is always going to have that “this is how it is” tone — but it’s really “this is how I see it,” not “this is how you should think.” And I really believe that’s the key disagreement: people getting their dander up over being told what to think, when that’s not what the review is doing at all.
@Matthew A and @River: Also, on the topic of graphics, I would offer that a game like SFIV, with a surprising art design, the graphics did significantly color my experience and my resulting review. They went for something specific and succeeded (in my eyes), and I really enjoyed that as a major component of that game. When a game’s goal is simply “realism” then yeah, it’s sort of like “well, the graphics look…real.” And some games, even if that’s their goal, look better in motion than the still screens suggest. I think people expect me to talk about the graphics so I do, just like sound and control and stuff like that — but I don’t have a checklist, and I don’t have to do 100 words on graphics, 100 words on sound, etc. I find the elements that define the game or make it unique, and I spend more words addressing those.
@Matthew A: “I think there’s a sort of psychological pressure to not review a game below 7 or so unless it’s REALLY bad.”
I think the weirdest example of this is Game Informer’s scale calling 7 an average game. When it comes to the 10 scale, I think the average is 5. Mathematics tends to agree! But there is that “7 is safe” mentality and it’s hard to break out. Eric Bratcher (Next Generation, PSM, and currently Executive Editor at GamesRadar) put it very simply: “A game doesn’t start at 10 and slide down the scale, the way a fan approaches it; a game starts at 5 and works its way up or down based on the reviewer’s experience.” I like going in more or less neutral — okay, it’s a game, and it runs on this machine. Now what? It’s probably “okay” because it got published. Is it above that average? Below? I really like that mental approach.
When I start doling out 2 (Space Giraffe XBLA), 3 (Track & Field XBLA), and 4 (Monopoly Xbox), it’s because I’ve really found something I feel is fundamentally wrong with it — a design choice at the beginning of the development, or a missing element, or some majorly failed implementation. And when I’m looking that close, there is totally a difference between 2, 3, and 4, and I really want to justify those choices and say something of value — really give it a critique. But I’ve also seen people use 2s and 3s as bitchslaps, so I think it comes down to personal responsibility of the writer.
And wow, I love the idea of turning off the scores.
Comparatively, Dan, when you’re dolling out scores that are that low, 2, 3 and 4, do you expect the reader to filter between them? When I see a game that scores low, Monopoly at a 4, I don’t equate it with being a bad game, but rather something that is now entirely off of my radar. Do you think that, at that point, readers don’t care if a game below 5 gets a 1 or a 4? If something is fundamentally broken, it can fall in to any of these categories, but it’s what it either does wrong that makes it slump further, or what little it does right to boost it up. But if at its core it’s simply shit, at what point does it matter whether or not an irredeemable flaw is associated with a 2 or 3? Either way, the majority of the audience recognizes it as trash, and that they don’t need to think about it.
On that note, one thing I always liked about OXM is that you guys have always used just about the entirety of your scale. I’ve seen 2s through 10s, and everything in between. The notion that review scales start at 6 and end at 10 doesn’t exist with OXM (or PTOM, for that matter). I like that. I just find it strange when we’re in territory as low as a 3, and whether or not it really, truly matters whether it’s regarded as “Broken,” “Wounded,” or “Deficient.”
Strangely, I don’t feel the same about it when we get to 8s, 9s, and 10s. Bizarre.
Dan:
“I would argue that reviews don’t need to change, but the audience needs to change in how they perceive them, for exactly the point you cite.”
I completely agree with this, but in this case, do you think the institutional mindset has done a disservice to that end? To me, it’s easy to say “well, people need to change how they think about reviews” and they do, but the way reviews are presented in many publications seems to engender this misperception.
Dan said:
“When I start doling out 2 (Space Giraffe XBLA), 3 (Track & Field XBLA), and 4 (Monopoly Xbox), it’s because I’ve really found something I feel is fundamentally wrong with it — a design choice at the beginning of the development, or a missing element, or some majorly failed implementation. And when I’m looking that close, there is totally a difference between 2, 3, and 4, and I really want to justify those choices and say something of value — really give it a critique. But I’ve also seen people use 2s and 3s as bitchslaps, so I think it comes down to personal responsibility of the writer.”
With regards to approaching it with a neutral, score-of-5 mindset, I think that’s a good methodology, but it’s got to be difficult to present that weltanschauung to the reader when for many 7 really is the average. I don’t read reviews so much any more, but I spent probably 15 years of my life doing so, and most publications over time have tended to skew to about the same scale as GI. I think most readers have had the same experience. That’s some of what I’m referring to w/ that mental calculus people perform when they read a review score. I might know that 5 is the average on a scale of 10, but if I’ve read 500 average reviews in my life with a score of 7, I start translating scores into what they mean to me. I like your approach better, but does it really translate readily to the jaded review-reader?
You could always reset their expectations by explaining your method as you did above, but it’s hardly something you could do on every review. The only way to completely avoid what is essentially a miscommunication (your 5 is a GI-esque 7) is if it’s part of the methodology for your entire publication, and then make it a point you belabor in your reviews section every month. Change the culture, essentially. Otherwise I can’t help but suspect that you get more than the average # of upset responses to a review with an “honest” 5, equivalent to the skewed 7 that would be awarded by a similarly minded reviewer at another publication. This exerts more upward pressure on review scores.
I don’t read reviews much anymore, but it sounds like maybe this is starting to change for some publications, and I call that a good thing.
I don’t know if I agree though with anything below 5 representing something fundamentally broken w/ a game. To my mind, if the game has something wrong with it which essentially alters the experience so wildly, then its a 0. At my work for our performance reviews we have a score called a zero default, which is different from an actual 0. It represents something so fundamentally wrong that it shoots down your score for the entire quarter (punching your boss, for example, or hanging up on a customer). I would make the case that if a game is either so busted as to be basically unplayable, or otherwise is so wildly different from the advertised experience (the car I bought is actually a banana sundae, etc.), then its a zero default. Fundamentally broken.
Another alternative to game scores would be to rate games based on a scale that is one hundred percent inscrutable. Here’s my recommended scale, in random order.
Apple
Parked car
Daisy
Bottle
Moon
Beetle
Fragrance
Jumping jacks
Ink
Mangrove
Shoe
So, for example, the new Prince of Persia is more Shoe than Moon, and here’s why… etc.
The reader then HAS to read the review to get any idea of the reviewer’s experience.
That is fucking brilliant.
Dan also said:
“I would argue that reviews don’t need to change, but the audience needs to change in how they perceive them, for exactly the point you cite.” (Thank you Steve for reminding me…)
Unfortunately it’s difficult to make a strong case for your customers changing your expectation. I can’t help but be reminded of the ZP review of Fable 2, and the point (however snarky) he makes about adjusting your expectation before judging the game.
The essential choice with review scores, I guess, lies in the dillema of matching the readers expectations with regards to the skewed score, or defying them and slowly changing the status quo.
Or get rid of them altogether. But that can’t happen until something better replaces their essential function.
There will ALWAYS be preconceptions, and if you say you go in to a game without any kind of idea what you’re getting in to (Unless you have literally NO IDEA) then you’re a god damn liar. I went in to Red Alert 3 knowing NOTHING about it because I forgot it was coming to consoles and didn’t have an interest in the PC version. I was clean slate, but still had expectations because I don’t like RA or RA2. But I don’t think that had an effect on my review.
On the other hand, I went in to Vigi 8 Arcade with huge expectations. They weren’t met. That didn’t factor in to the review, because it was simply an OK game, but you can’t help but feel that remorse.
@Mitchy: “But if at its core it’s simply shit, at what point does it matter whether or not an irredeemable flaw is associated with a 2 or 3?”
The proof’s in the text. Yeah, some people are going to look at the scores and nothing else…but I’m not writing words for them, right? If you don’t care about the difference between a 3 or 4, then you probably won’t read the review (unless you hope to see someone cleaning their claws). But if you really do care to know, I will always explain. (For the record, SG’s 2 was because of its counterintuitive mechanics and lacking the ability to teach the player how to play, even with a tutorial; Track & Field’s 3 was for not providing the original control scheme; and Monopoly’s 4 was for being a Wii port poured into an ill-fitting 360 mold. Each sin is slightly less serious, and other elements of those games were polished or acceptable.)
@Matthew: “The only way to completely avoid what is essentially a miscommunication (your 5 is a GI-esque 7) is if it’s part of the methodology for your entire publication, and then make it a point you belabor in your reviews section every month.”
Agreed, and we do. I’m really proud that we’re adamant about running our complete chart, with a one-line explanation of each score as well as a word (“Broken, Mediocre, Great, Deficient”) to help make that number far less arbitrary. Whether you agree with our score or not, you know where we’re coming from when we assign it. We refer to this chart internally every issue, too, and we will shortly when we have our issue score meeting today. I’ve also blogged about it on our website to bring it up in a different context, and to explain it to people who don’t read the mag but do react to the isolated scores they see posted on Metacritic. (For the record, here it is: http://www.oxmonline.com/article/xbox-soapbox/how-not-read-oxm-review)
@Steve: “do you think the institutional mindset has done a disservice to that end? To me, it’s easy to say “well, people need to change how they think about reviews” and they do, but the way reviews are presented in many publications seems to engender this misperception.”
Yeah, absolutely — we’re pushing a rock uphill, and it feels like gravity is helped even further by a whole bunch of people pushing on the rock’s other side.
Inertia is hard to change, but the truth is we have always reviewed things the same way. I can only address and ensure internal consistency in our review methodology and try to lead by example; I cannot explain away what other publications do…even though what they do DOES color how people see our reviews. So, I guess it goes back to us printing that chart every month and saying “here’s our thought process, laid bare.” Yeah, the reviews still have numbers, and we have talked about what would happen if we threw them out altogether — but the very loud feedback is that the bulk of our readers find scores valuable, even if they frustrate the hell out of us when they are misinterpreted by the people who don’t seem to understand their true purpose.
Part of me does want to be that nasty parent and say “Look, if you can’t use this responsibly, I’m going to take it away from you entirely.” But then we’re not giving the bulk of the audience what they want/find valuable because of the rogue actions of a few hotheads and nitwits.
And as for changing the mindset…well, so far my best weapon is rambling blog posts.
BTW, I totally would have given Fable II a parked car, but the short storyline ultimately made it a fragrance for me.
“BTW, I totally would have given Fable II a parked car, but the short storyline ultimately made it a fragrance for me.”
That was ace.I am pro of this rating system.
Dan said:
“BTW, I totally would have given Fable II a parked car…”
So would you give it a hummer, or…?
*dodges tomatoes*
Before that, Dan said:
“I’ve also blogged about it on our website to bring it up in a different context, and to explain it to people who don’t read the mag but do react to the isolated scores they see posted on Metacritic.”
I didn’t think of that. Metacritic actually makes me a little uncomfortable, with regards to game reviews. Industry-wide, rating scales are so inconsistent (the whole 7 vs. 5 thing we’ve been talking about) that I can’t help but be concerned about the actual math here. The biggest problem I see is that the aggregate scale used by Metacritic will change from game to game (based on which publications’ scores are included), so that internal calculus I’ve been nattering on about, the windchill effect, gets buggered. It’s hard to adjust for what your own tastes and experience are when, in that given source, a 5 for one game could actually be the same as a 7 for another. At least w/ individual publications there’s usually some consistency.
This is aside from the essential problem of boiling down a multi-hour interactive experience into a bald number. (More a problem @ Metacritic, since the score is the highlight, and all the text is fluff.
That brings to mind another argument against scores in general, even the buy-it, rent-it, fry-it arbitration; they make sense in movies since the range of things to critique is pretty limited, but games (and sometimes an individual game) is so much more diverse that it’s just a lot harder to apply the number. In that respect it’s not surprising that scales are so diverse.
It was in vogue for a while to score different components of games (graphics, sound, replay, etc.) and this was somewhat more useful, but it faced a huge flaw of its own. Not all games are equal, and great graphics may really make one game while being totally irrelevant to another. Symphony of the Night was a wonderful game, but the experience was honestly enhanced because it’s just so damned gorgeous and well-animated. Meanwhile, Final Fantasy VIII was stunning visually, but (except for a certain class of gamer), it was pretty sub-par, with its bent magic system and overall broken battle system. Also a plot that refuses to get moving at all until halfway through the game, like that boulder Dan’s loving on.
Reviewers who don’t want to use scores could probably get a lot of mileage out of getting better at writing headlines for their reviews. Good/bad/ugly doesn’t hurt either in the absence of a score. Scores, and these replacements, are all about setting the tone; flawed as they are, they’re still an essential part of the communication. They give context and set the tone of the review, and the void has to be filled or the review struggles.
Blog reviews get more mileage out of straight text, however, due to the opportunity for dialogue.
Jesus, why do I have so many words!