As a primer for our upcoming GDC talk, Peggle Blast! Big Concepts, Small Project, PopCap Team Audio has written a short blog series focusing on different aspects of the audio production, covering real-time synthesis, audio scripting, MIDI scoring, and more!
Jaclyn Shumate – Realtime Synthesis for Sound Creation in Peggle Blast!
https://www.audiogang.org/members/jaclynshumate/articles/
RJ Mattingly – Peggle Blast! Peg-hits and the Music System
https://www.audiogang.org/members/rjmattingly/articles/
Guy Whitmore – Scoring Peggle Blast! New Dog, Old Tricks
https://www.audiogang.org/members/gwhitmore/articles/
Scoring Peggle Blast! New Dog, Old Tricks
Flexibility and dynamism at the musical note level will be necessary to advance game scoring in the 21st century. Without that level of detail and granularity (at runtime) we’ll be stuck painting our scores with a wide brush, and no ability to embellish the finer details of gameplay. More than a nifty way to save lots of memory, note level dynamism also enables harmonic, rhythmic, timbral, as well as melodic flexibility. MIDI happens to be the most prominent, and longest standing commercially available format for controlling music at the note level, but it’s not the only method. And for various reasons MIDI game scoring has been maligned over the years, despite there being a rich history of MIDI-type game scores past and present.
One reason this negative attitude developed was the adoption of a 1990’s technology called ‘General MIDI’ (GM). General MIDI sonified many a game score, even as many composers, cursed its very existence. Why? Because, despite certain inherent advantages, General MIDI used instrument sample sets (a standard set of 128 instrument patches) that resided on the gamers’ PC. While saving huge amounts of memory for the game, the instruments sounded overtly generic (by definition) and there was wild variation of timbre and quality across PC sound cards; from 8 voice FM only cards, on up to the 128 voice Roland Sound Canvas. So it’s no wonder that MIDI, as a game scoring delivery format, still makes many composers and developers cringe. But in our rush to abandon MIDI game scores in favor of fully rendered PCM/wave mixes, we’ve thrown the baby out with the bathwater.
20 years later, a convergence circumstances allowed the audio team at PopCap to revisit the concept of real-time MIDI in games. Audiokinetic announced the addition of MIDI functionality in its interactive music system, just as we received the edict from our producers to stay under 5MB for all audio in Peggle Blast! (iOS/Android). It would’ve been too easy to fall into the common practice of game music on mobile; create one main looping piece of gameplay music (30seconds to a few minutes in length), along with a small handful of short intros, ends, and stingers. Then we could sulk in the corner and say smug things like ‘Audio never gets the love’ or ‘Oh well, no one listens to mobile games with audio on anyway…’ (this last quip I’ll take up fully in an upcoming blog post). Instead, with the team’s blessing, we took a risk with the Beta version of the MIDI tech and the adventure began.
There are a few keys to making MIDI scores that are both dynamic and aesthetically pleasing (as well as consistent):
- Creating custom instrument sample banks: Using Wwise’s software sampler, I custom built a set of sample instruments from orchestral recordings done the previous year. As these instruments were specifically for Peggle Blast they were tailored with it’s music aesthetic in mind and even helped to define that sound. Also using a software sampler in-engine, ensured that all players, on all platforms, would hear a consistent score.
- Instrument choice/orchestration: With limited space, staccato orchestral samples are both versatile and short! Likewise, pizzicato strings are short samples that can create varied textures. Working in tandem with sustained loop-based samples, a multi-sample layered effect was achieved. These choices helped us work within the technology limitations rather than fighting them.
- Real-time DSP: Each sample instrument uses effect sends to feed the real-time reverb in Peggle Blast. This helps bring the sound into a single space, while making transition points smoother as reverb cascades over the boundaries. The more layered and granular the score, the more aspects of its mix needs to be authored within the game audio tool/engine.
- Combining technologies/techniques: Blast’s in-game score is pure MIDI/samples until a game is won, which triggers the (PCM/voribs) full stereo orchestral recording of Ode to Joy. Wwise also allows simultaneous/synchronized playback of MIDI and PCM tracks, allowing for a truly hybrid, best-of-both-worlds, range of possibilities.
Peggle Blast’s Music Design
Music progression occurs after any ball shot is complete. This gives the game a sense of momentum as it’s being played. I wanted to create a harmonic shift between shots to emotionally lift the player. In film/TV, harmonic shifts are a common tool to assist scene transitions, so I treated each shot/turn as a scene transition. This also allows the harmonic development to move at the speed and pace that the gamer is playing. By contrast, the pacing of a pre-baked loop would be the same regardless of a players’ style.
This led to a chord map and music segment flow chart that functioned analogously to a spotting session/cue sheet in a film or TV show. Cue sheets in film can have detail down to the second and frame, and my goal was to approach this level of detail in Blast. The world menu and UI flow are largely in D major and A major. A game session kicks off in D then moves up by a fifth after each ball shot, thus following a circle-of-fifths in reverse over the course of a game session (a circle of 5ths typically follows an up by a 4th or down a 5th pattern, but I found in this case that the up a 5th pattern gave the score more of a surprise/ah-ha feeling shot-to-shot ) A game session has both an indeterminate number of balls/shots to begin with and an unknown number of turns to either win or lose the game. Thus the circular harmonic pattern (12 music segments; one for each key center) allowed the pattern to continue even if a game was more than 12 shots long.
With one peg or one ball left, the flow moves to A Major (the penultimate segment) to setup the D major Ode to Joy or the loss segment. The trickiest part here, and an area I’d like to improve upon in upcoming projects, are transitions between the more distant key centers. For example, the game can go from any of the 12 key centers to A major, and some work better than others. A useful technique for the more distant transitions is to have a transition segment between the start and destination keys. For the most part these weren’t needed, but the use of a few more transition segments would’ve made some scenarios smoother.
Peg Hits combine 2 sample instruments; harp for any peg that is hit, plus a marimba layer if an orange peg is hit. This adds a bit of an accent as the crucial orange pegs are taken out. SFX include the physical impact, plus any materials such as ice, sticky mud etc. Pegs follow the scale tones that are defined by the underlying music segments. For example, in Wwise I simply drop a marker tagged DM, if I want a D major scale to play over the segment. This is also where I define the low and high range for the peg scale (using MIDI note numbers). The PegHit script, written by RJ, handles the rest and plays the correct series of notes.
Stingers layer over the pegs and main music segments to accent specific game events. When a phoenix egg is hatched a brass riff plays as the bird flies away; The choir sings a sus9 chord when a free ball is earned. And added to the game in a recent update, the choir sings exclamations such as ‘awesome!’, and pegglicious! when spectacular shots are made. Of course these stingers have to work harmonically and rhythmically with the underlying music. Since every major key and several minor keys are used in the score, MIDI worked perfectly here. Each stinger type had between 12 and 15 MIDI files (one for each key center) and were matched with the appropriate music segments.
Compositional Work-flow
The compositional work-flow was unique for Blast due to the custom sample banks and dynamic scoring. After an initial sketch it’s important to compose with instrument sets that will be part of the final mix. So I setup Wwise and ProTools to talk with one another, allowing me to play/record MIDI in ProTools while triggering the custom sample banks constructed in Wwise. (the lync between Wwise and PT is far from perfect at this time, but the fact that it’s possible is a huge step in the right direction) After segment mixes were finalized, the MIDI files (one MIDI file per Wwise segment) were exported from PT and imported into Wwise segments. If a change was needed, I’d edit the MIDI in ProTools and re-export right into Wwise. For future projects, I’ll be working with Nuendo rather than ProTools due to MIDI handling and export functionality.
More Than a Memory
We started using MIDI and sample banks due to an insanely small memory footprint, and it served us well in that regard with over 30 minutes of music in game (and more could be added for mere kilobytes per minute). Yet MIDI’s note level dynamism is what gives it power well beyond the memory savings. This is why, even with memory rich console titles, I plan to utilize Wwise’s MIDI capabilities in any game that could benefit from detailed scoring. MIDI is just a tool, and like any tool it can be wielded with precision or incompetently. There’s not an ‘either-or’ choice between MIDI and PCM/wave. I predict the best adaptive scores will combine and make great use of both methods and more.
