TIMING IS EVERYTHING!
When a composer scores a film, the visual timing is locked, and each moment of the film is forever linked to a specific moment of the music. This allows the composer to craft the score to flow purposefully with the visuals, sometimes synced very tightly to visual cues (Carl Stalling) or change harmonies on picture edits (John Williams) or more loosely set to picture. Bur regardless of the scoring aesthetic, this range of tight to loose scoring is purposeful and precise in the film world. In other words, it’s no accident. And if you want to hit a very specific frame with a music event it’s not only possible, it’s often expected.
Present day game scoring is still quite loosely tied to the game visuals, not necessarily by aesthetic choice but by supposed technical limitations, lack of will (from composers as well as game developers), not to mention the unique compositional challenges game scoring presents. Yet, if we, as game composers, ever hope to match or exceed the artistry found in film scoring, greater more nuanced adaptive techniques must become more commonplace. Timing IS everything in storytelling, and the only way our game scores can follow (or lead) a game’s non-linear visuals and design is through the precision and flexibility of our music.
WHAT THE *&^% IS ADAPTIVE SCORING ANYWAY??
There is not a single method, technique, tool, that defines adaptive music. Especially now, the field is wide open to invention and creativity that will result in a myriad of methods and ways to approach game scoring. Peggle 2’s score is just one way in which the game could be scored. I view adaptive music as a spectrum from less flexible (3 minute looping wave files) to systems with greater flexibility (smaller phrases, instrument stems, samples, MIDI, real-time synths etc.). There is no correct place on this spectrum; it’s simply a range of possibilities (and tools).
BREAKING IT DOWN
In order for a score to have flexibility, the musical elements must be kept in their elemental form within the audio engine. Again there’s a spectrum here; for linear cut-scenes a fully baked wave file will work fine, but in order to match the non-linear nature of game-play, music must be broken into component parts.
This is why for Peggle 2 we recorded most of the score one orchestral section (violin 1, cello, clarinet etc.) at a time, and recorded individual phrases, sub-phrases, and even individual sample notes (for Peg Hits). These various puzzle pieces were then edited, mastered (light EQ, compression, using Neve and 1176 plug-ins in Pro-Tools) and imported into the audio engine (Wwise).
AUDIO ENGINE AS DAW
Once in Wwise these pieces where reassembled in various music segments/tracks, phrase by phrase. While starting with the originally composed phrases, I often rearranged the elements to create variation, and extended phrases, as well as the ‘Idle phrases’ which occur if the user is inactive for more than 30 seconds.
real-time DSP, particularly music reverb plays a few important roles. First the reverb allows for smoother more natural sounding transitions
(baked in reverb and reverb tails can make transitions sound stilted, making the seams between them more obvious). Second, real-time reverb and mastering of the mix, allows for the score to be mixed within the context of the game itself, rather than offline. Many game scores are mixed more for the soundtrack (higher compression, etc) and that can make the music sound as if it’s sitting on top of the mix, rather than being part of the overall sound mix. We were able to experiment with reverb settings and amount, as well as dynamics (and found we didn’t want to over compress the music in-game) until it sat well with the rest of the sound elements.
TRANSITIONS
In film scene transitions are nuanced, and while they often change moods completely, it feels like the music is continuing, which helps the film’s sense of continuity. The most common transition in games today is still sadly, the crossfade… And sometimes mute/unmute of music stems; still relatively basic by film scoring standards. (of course there are a good many exceptions, but we’re still at a place where these are exceptions rather than the norm)
The score for Peggle 2 gameplay is broken up into 7 ‘phrases’ or sections, plus the finale. These phrases follow the progress of gameplay, regardless of the length of a given game. To accomodate this, each phrase can go for an indeterminate amount of time (while not statically looping) then transitions on musically defined boundaries (usually 2 to 4 measures) to the next phrase in the progression. This approach allows for harmonic changes, tempo changes, thematic changes, as well as instrumental changes on each of these boundaries. Aesthetically the goal is to create an emotional ‘arc’ to the gameplay, regardless of gameplay style/pace.
PEG-HITS BECOME THE MELODY
When the ball hits a peg in Peggle 2, it might be considered a sound effect, and it is. But it’s also part of the music. We built the system so that peg-hits created an ascending diatonic scale which changes to match the harmony of the underlying ‘phrase’ of music. This proved trickier than first conceived. Each of the 7 phrases is assigned a specific diatonic scale (26 notes up or 3 and 1/2 octaves) and that scale changes if the music progresses to the next phrase i.e. phrase 1 may be linked to a G Major scale for peghits while phrase 2 is linked to an A Major scale.
The challenges:
If the music moved from phrase 1 to phrase 2 during a shot, the scale needs to change during the shot as well, while still continuing to ascend To accomplish this we tracked two RTPC values; peg-hit number from the start of a turn (so that it always ascends) and the current phrase (so that the correct scale plays). Yet when phrase 2 is called, it may take a few seconds for it to reach the transition boundary and begin playing phrase 2. Therefore we needed a callback from the audio engine letting the game know when to really start the next peg-hit scale., therefore matching the start of its underlying phrase.
All this points to the thorny challenges that harmony and harmonic changes present for adaptive techniques in general. But harmony must be dealt with head-on in game scoring if we are to break old patterns of monotony (there are many mono-harmonic scores out there that do so to avoid and work around these challenges.)
THE ROAD AHEAD
Over the past 10 years or so, game music has made significant leaps. Live orchestra is no longer a novelty but common, often using the best for hire orchestras in the world. Surround is a given on console and is used to great affect in games. It’s arguable that many game scores are reaching the caliber of film scores in sheer compositional prowess. What’s left? Timing timing timing!!! Transitions, transitions, transitions!!! And the only way to get there is to expand and evolve our creative and technical tool-chest, with adaptability and flexibility at the forefront.
Gameplay captures:
http://www.youtube.com/playlist?list=PL35MOja90e7_cuosJzgYh59jWVmK9KLHE
