We've all played a game at one point and asked ourselves, “What were the developers thinking? What's meant to be enjoyable about this?" Often, a game just isn't for you, but sometimes, it simply isn't for anyone, and ideas which sounded fine to the developer during production just don’t resonate with the game's target audience.
One of most common causes of this is that many developers simply have no idea who is going to be playing their game. In fact, it’s common for games to be made based on what studios think players will enjoy - based, more often than not, on the developers' own tastes.
Perhaps this stems from the origins of our industry, with games being developed in bedrooms by one person, making a game because they themselves want to play it. Even with the larger teams that we have today, it's often the case that key decisions are being made by only a select few in the studio. Is their opinion really any more valid than anyone else's?
And opinion is all it is, most of the time. Sure, we all have experience, but when it comes to designing successful games, what's really needed is not opinion, but evidence that the design is sound. It's all very well coming up with a game concept, but if you want to increase the likelihood of success then evaluating designs before release would seem like an intelligent approach. This evaluation of creative content is precisely what happens in the other entertainment industries, yet somehow, videogames have been slow to embrace these methods.
Movies
The movie industry has been evaluating what audiences think about upcoming releases using test screenings since 1928. A representative audience is recruited, and screenings of the full movie and alternate endings are shown. After the credits have rolled, questionnaires may be filled out, audience members may be selected for interview or the director may review video recordings of the audience to watch their reactions at key moments.
Does this process improve the movie experience? Undoubtedly. Many movies have been improved by changing actors, tweaking narrative, creating space for the audience to react, building tension, or balancing engagement levels. Movies such as Jaws, Shaun of the Dead, and 28 Days Later were all improved by test screenings. During screen tests of Johnny English, problems were identified with the introduction of the main character. The beginning of the film was re-shot with a focus on addressing this issue and the resulting test scores increased considerably.
It’s also often the case that once a director has made an edit based on results of a test screening, they are often surprised at how badly they made the original edit in the first place. Even experienced directors admit that their creative output needs validation: Woody Allen says that it's easy for him to write a script, but he's no idea if it's any good until he sees an audience's reaction. This inability to see our own mistakes is something I’ve mentioned before, and it’s as true for videogame development as it is for the movie industry.
TV
The TV industry takes the creative evaluation process one step further and performs realtime testing of comedy material during live recording. During a recent tour of the Warner Bros studios, I was told how they record Big Bang Theory to ensure the best possible outcome. After a particular take, the director may say to the audience that the cast are going to try that scene one more time, pretending that he didn’t get the shot he wanted.
What’s really happening, however, is that the actors are going to try out a different joke. This could happen multiple times and the joke with the biggest laugh will be the one which makes the final cut. When aired, the viewer thinks they're watching one carefully crafted show, they're not. What they're experiencing is the best possible permutation of jokes as rated by the test audience on that day of recording. The episode that ultimately airs has been optimised thanks to audience feedback.
Games?
Both the movie and TV entertainment industries evaluate their content for the same reason: to give themselves the best possible chance of creating a great experience. They openly admit that they're not sure if their creations will be enjoyed by others, so they put processes in place to minimise the massive risks of getting it wrong.
What these other industries have learned from experience is that involving their target audience in evaluation produces better results than if they ignore them. For our industry, then, opportunities for both better and more original games exist if studios are prepared to embrace them. But are their arms - and minds - open?
Comments
2But I want good games, not successful ones. And to be honest the best and most original games of this generations were the result of studios just saying "let's stop giving a f*ck and just do this because we want to". The moment you tailor the experience to the audience you get interactive movies like Call Of Duty because there were a few too many 12 year olds crying for a lower difficulty and more explosions. Just like the movie industry ends up with boring Transformers flicks when they do that. Sure, they're profitable. But would anyone right in their mind connect these franchises with the words "unique" and "refreshing"? Do you, Graham?
Edit: Also, isn't originality a success in its own way? Profit isn't the only way to measure a game's quality, and many would in fact argue that it's the wrong way.
With about a century of artistic development in film and television, the thing you think video games could learn from is focus testing? The most cynical, money-grabbing, integrity-lacking practice the creative industry has seen?
What about important things like character development, dramatic arcs, screen theory, and stable production?
The last thing we need in video games is more generic mass-market nonsense.