As a primer for our upcoming GDC talk, Peggle Blast! Big Concepts, Small Project, PopCap Team Audio has written a short blog series focusing on different aspects of the audio production, covering real-time synthesis, audio scripting, MIDI scoring, and more!
Jaclyn Shumate – Realtime Synthesis for Sound Creation in Peggle Blast
https://www.audiogang.org/members/jaclynshumate/articles/
RJ Mattingly – Peggle Blast! Peg-hits and the Music System
https://www.audiogang.org/members/rjmattingly/articles/
Guy Whitmore – Scoring Peggle Blast! New Dog, Old Tricks
https://www.audiogang.org/members/gwhitmore/articles/
Real-Time Synthesis for Sound Creation in Peggle Blast!
Recently, on Peggle Blast, PopCap’s newest mobile addition to the Peggle Franchise, we were challenged by a directive from the team to keep all of our audio content under 5 MB. Having little to no memory is an Audio Designer’s worst nightmare, and we were concerned that we wouldn’t be able to come close to hitting the quality bar we had achieved on our last Peggle project, Peggle 2. To give some comparison, Peggle 2 shipped on Xbox One with 783 MB of audio.
To work around the lack of available memory, we used a combination of MIDI and real-time synthesis for audio creation. We built sounds using tone generators and Digital Signal Processors (DSP) within the audio engine (Wwise), while matching sounds directly to visuals, real-time, in the game engine (Unity). We didn’t work in the traditional way of creating content, where you take game capture and then make sounds to picture within your Digital Audio Workstation (DAW). This new workflow allowed for an immersive development experience, a tighter feedback loop, and a more dynamic product overall.
Inspiration came from classic games like Super Mario Bros., The Legend of Zelda, and Pac Man. These games were able to achieve iconic audio despite having limitations. In fact, these constraints contributed greatly to their signature styles and may have boosted creativity. On Peggle blast, the constraints shaped our daily goal of creating sounds without using any memory. Memory hogging .wav files were reserved for a few iconic Peggle sounds and specially curated .wav file building blocks that we could manipulate in varied ways to use as layered elements with our generated sounds. In the end, we were able to ship Peggle Blast with only 1.3 MB of sound effects and 3.5 MB of music, yet it contained hundreds of sound effects and over 30 minutes of interactive music.
The Workflow:
Let’s say I wanted to make a sound for a splashy visual effect that burst across the screen when the player achieves a high score, and I wanted it to sound like a sweeping bleep bloop with magic sparkle dust added in. The first step I’d take would be to make an approximation of the sound within Wwise. Let’s say in this case, to create the bleep bloop, it’s a square wave, pitched up over .5 seconds, created using Wwise’s tone generator with various delays added. Then, to get some sparkle dust, I could add in a swooshy sound created in SoundSeed Air, with tremolo and more delay. By taking generated sounds, experimenting with playback behaviors, DSP, and layering, very quickly a unique sound would be born.
Once I had the basic framework for a sound that I was happy with, I would connect Wwise to Unity via the Wwise profiler, view the visual effect real-time in game, and continue to tune the sound until it fit the gameplay sequence I was creating it for. It was real-time sound creation with real-time sound generation!
We spent a lot of time exploring different methods to expand our pallet of sounds with our limited toolset. There were a few of us working in the project at various times, and we borrowed liberally from each other’s tricks. Someone would come up with a new way to get a different sound, and then that technique would proliferate into other sounds throughout the project. Additionally, effects like escalating pitch over time on the same sound were a big win for creating sound variability and excitement, without requiring a new asset.
The Results:
There were a number of interesting outcomes of working this way, beyond the memory savings. Artistically, creating sounds with such basic and pure elements was a fun and refreshing exercise in sound design. I was entirely forced away from using real-world assets and literal sound design. Technically speaking, not needing to use a DAW to create sounds made for a faster design process. No time was spent creating game capture, importing it into the DAW, exporting your final asset, and repeating that process again if edits are needed. The DAW was no longer an extra layer between the sound designer and the game, and it felt really good. We were more closely integrated with the game than I have ever been able to work in the past.
With real-time sound generation, the sky was the limit in terms of interactivity. All sounds could be stretched, pulled, pitched, or delayed in whatever way worked best to match gameplay, and it was truly dynamic. The audio content was a living, malleable, part of the gameplay. With RJ Mattingly down the hall handling all of our scripting, it was quick work to add Real Time Parameter Controls or whatever else we wanted to use to push the interactivity. Another enormous added benefit was that when art or design changed an asset, it was fast and easy to iterate on the sound and make it match the new timing. You could just jump into the audio engine, change a few numbers that controlled the length of a sound, all while viewing the asset real-time in game to make sure it fit the visuals (can you imagine being able to do that on a large console project? It would be awesome).
Working on this game became an inspiration for what can be accomplished with truly generative, dynamic audio. A tight feedback loop starting from sound creation through gameplay made for a great feeling workflow. The limited toolset forced us to hone in on how we could maximize the effectiveness of audio content without increasing the size. Peggle Blast was a good reminder that creativity can flourish under constraints, and I can’t wait to keep exploring real-time sound generation techniques on future games, both large and small.
