Mark Kilborn has one of the greatest jobs in the world. As the audio director of Raven Software, he's been involved with some of the most popular games of the last decade, Call of Duty included. We sat down with him to talk game audio, virtual reality, and why Mario's coin sound is one of the best game audio sounds ever.
(OSSIC) Can you give a brief background about work and how you originally got interested in audio?
(Mark Kilborn) My interest in game audio developed at a very young age. I was 8, I was in love with NES games and music. I played Metal Gear for the NES, and it just clicked: "Someone is making sounds for these games. I want to do that."
The interest stuck. I played in bands as a teenager, started DJing professionally around 20, got into audio post production at 23 I think, then jumped to games at 25 (in 2005). I've spent the last decade working in games, and have been involved with projects like Forza Motorsport 3, Borderlands, Project Gotham Racing 4, Brothers in Arms: Hell's Highway, and most recently I've been involved in the last seven major Call of Duty titles, including the Call of Duty: Online free-to-play shooter in China.
What are your top 3 audio moments in video games? Your work or the work of others. What made them great?
(MK) This is a tough question, because it depends on the criteria. I find things like the Mario coin collect sound, Megaman teleporting, Pac Man eating a power pellet, and other sounds like that to be iconic. They're burned into our collective psyche. You can play these sounds to most Gen Xers and just about any millennial and they'll instantly recognize it.
Then there's the modern perspective, games that sound amazing by today's standards. I find they're not nearly as iconic, but they're powerful in a different way: in their power to immerse people in an experience. Red Dead Redemption is far more immersive than Mario, but I can't pick out a single sound from that game and say "That's a sound I'll always remember."
By the modern standard, I'd pick the following:
Red Dead Redemption. A few moments in this game. The first gunfight you get into. The crossing into Mexico has my favorite use of licensed music in the last generation of gaming (See video below).
Call of Duty: Advanced Warfare. I didn't work on this part, someone else did, but the first mission of the game, Induction, is a master class in immersive sound design. The guys who did it did an absolutely incredible job. The bit where the robot tank crawls over the trench while you're walking underneath is the highlight for me, but the whole thing is just epic. I'm regularly in awe of the people I get to work with.
Limbo. There are so many little moments in this game that stand out to me. The neon signs flickering on for the first time. The river crossing. This game was on a much smaller scale than the two above, but it's no less powerful for it.
To follow up on the previous question, In your mind what defines a better game audio experience in one game over another?
(MK) For me it's a matter of how effectively the audio contributes to the experience of the player, and it's a very subjective thing. It's hard to say "here's a checklist of what's required to be great." The needs for a game like Call of Duty are very different from a game like Hearthstone, so the path to success for each of them is very different.
The soundbite answer is that the audio immerses the player in the experience and never distracts them from it. But that's just scratching the surface of it.
Let's talk a bit about your work. What game audio engines do you use most often? By choice, or as a company choice?
(MK) We use a proprietary engine for Call of Duty, all of it developed internally. No middleware, nothing that's publicly available. It's a choice of the audio teams involved in the various projects, but we're all in agreement on it.
What's your typical work process look like for developing great game audio? And what are the typical challenges you face?
(MK) This is kind of a huge question. It starts with figuring out the vision for the project, and then figuring out how audio contributes to that. The "audio vision," which is a totally silly oxymoron but I love using it. In the early stages of a project we've mocked things up in a linear DAW to try to get things sounding the way we want, and then dissected what we built and started thinking about how to implement the functionality in terms of systems.
Once you've got the audio vision nailed down, you start to think about what systems/features/pipelines/etc. you need to support that. In CoD we have the benefit of years of past development, so we incrementally build from the previous project. It's a huge undertaking to start from scratch and I haven't done it in about 6 years. But anyway, you get those planned out, you work with your audio coders to build the functionality you need but don't have (middleware is nice but it's never a one size fits all solution). This includes everything from pipeline for assets into the engine, implementation methods for various types of sounds, runtime mixing and DSP effects, etc. It's obviously a massive job with lots of moving parts. Project management skills are useful :)
Outside of the tech, you've got the aesthetic piece of it to handle. I find it's best to set goals then create what our art department calls a "beautiful corner," a small section of the game that sounds (or in their case looks) like what you want the finished product to sound like. For Call of Duty it's a section of a mission, with maybe one big event, a couple of weapons, a grenade, some foley movement. You get a nice little playable section, you tweak it until it's perfect. And then you go forward and spread that aesthetic throughout the rest of the game, constantly referring back to your beautiful corner to ensure you're maintaining that target. And, if you decide to make a change, you change the beautiful corner so your reference point is still relevant.
The daily work is a matter of looking at the moment to moment experience of the player in the game, figuring out what's needed to support the experience, selecting library material or recording new source material, editing/compositing sounds together, implementing them in a way that supports the gameplay, getting feedback (from people outside of our department and from internal critique sessions), and then iterating until it's right, all the while keeping an eye on the schedule to ensure we're not falling behind.
Toward the end you start looking at the project from a higher level, mixing things, trying to bring it all together. It's an organic process.
Sub question: On a more technical level, how much time do you spend sorting out the spatial size/reverb/decay etc, relative to picking the audio source files?
(MK) I don't have a good answer for this. The answer is that it depends. What is the sound for? Is it indoor or outdoor? What else is going on at the moment? Is the moment super busy or super quiet? Is this particular sound a background element or something key that the player must hear? Both are critically important, but the weight of one vs the other depends on the sound in question.
What inspires you in designing an audio soundscape for a game? Film, music, real-life, hyper-realism, visual tie in, storytelling, other?
(MK) Again it depends on the game. For Call of Duty it's definitely hyper-realism. We're giving players an adrenaline ride. We often say "Don't make it sound like it would really sound, make it sound like you would remember it," referring to the mind's tendency to dramatize exciting experiences. We choose the right moments to throw on the hyperrealistic special sauce, and outside of that we try to keep things very grounded in the real world so that the hyperreal moments feel more dramatic. It's all about pacing and dynamic range.
With immersive content on the rise (virtual reality, augmented reality, etc), what role do you think sound will play in the future of this content, especially in for gaming?
(MK) I think it will be huge, but I also worry about the delivery methods for it. It will be huge because sound is more immersive than the visuals can be. For all the love and money thrown towards visuals, audio is the only thing in a game or experience that actually exists in three dimensions. The picture, even if stereoscopic, is still just images on a flat screen. Sound can truly fly around the room (if you're using surround). So the power of it to immerse the player is massive, and should be used.
Delivery methods though. I know people are tinkering with HRTFs/"binaural" encoding, which is great, but the quality of the player's headphones will have an impact on how it's perceived. And you're removing the sound from the room. If you're using speakers in a room, then you've got the issue of the disconnect between the player perspective and listener perspective when it comes to spatialization, which can be a tricky thing to resolve. I'm not sure what the best solution is, but whatever it is needs to be easy for the end user. I worry about the complexity of VR for the customer and its potential impact on the market in general, and this is just another layer of it.
It's a problem to solve, but it's a really exciting problem to solve. I'm envious of those that are getting to work on it.
A big thanks to Mark for a taking the time to answer our questions. We'd love to hear your thoughts in the comments below!
Share with your friends!
If you enjoyed this article here are a few more recommended posts: