My name is Colin Northway, I work on the Museum of Other Realities, and one of my roles is sound implementation. Today I'm going to talk about sound implementation for Durk van der Meer's piece, Terrarium.
Like many pieces we show in the MOR , Terrarium didn't come with a full sound-scape. Few visual artists are also confident sound designers but sound adds tremendously to the VR experience. We like to add audio whenever the artist is open to it.
We work with A Shell in the Pit Audio who does all the sound design for the museum and also the pieces we add audio to. They jumped into VR audio design very early and are an amazing team to work with. For Durk's piece, A Shell In The Pit had very little direction so Em Halberstadt had to make decisions about what needed sound, what to highlight, and what everything would sound like.
Every art piece in the MOR goes through a draft staging period where we have the rough layout of the room and the artwork's position in it. From there we get feedback from the artist as we move towards final staging. This drafting process also gives A Shell in the Pit a chance to drink in the piece and get a feel for what the artist is trying to convey and how to compliment it.
In this case, Em decided to break Terrarium into two ambiences, a spot sound and one stereo-split music track:
The Museum of Other Realities uses Moona to manage all of our audio. Moona is a Shell in the Pit tool written by Maris Tammik and Nicholas Zhang which is especially good at managing sound for VR projects. It manages all our disparate sounds, sounds being on the other side of portals, walls occluding sound, a bunch of tricks for making conversation more natural, and a lot more.
Usually, my job of audio implementation is really easy, I just have to drag some Moona assets into the right place in Unity and let Moona do its magic. For Terrarium, we had a slightly more technical challenge implementing the ambiences between the top and bottom floors.
In Moona, sounds radiate from a point and can be heard anywhere within hearing distance. This means sounds are "spherical shaped". But the bottom and top of Terrarium meet in a line and are right next to each other. Meaning spherical sounds are hard to place to cover one floor without bleeding into the other.
The solution to this was to use box-shaped colliders to bound in the sounds. Moona lets us turn sounds on and off when the visitor's head is inside of certain boxes, for Terrarium I made one for the top and one for the bottom. These let the visitor hear the ambience slowly rise as they appear the piece but also flip back and forth more suddenly as they move up and down the stairs. To make the transition between top and bottom smooth, Nicholas Zhang added a fade transition between the two sounds.
Terrarium also has one of my favorite strange VR audio tricks in it with the left, right speaker tracks. Durk put a video preview of his personal project Maranga to the bottom room and added two speakers on the wall. Em decided to add audio to the Maranga video but to make it in stereo with different tracks for each speaker. This is a strange, wonderful thing. All the environment audio in the MOR is spatialized. This means it's mathematically transformed to sound as if it's coming from behind or beside or above you. Stereo sound is part of spatializing but not the whole equation. In the bottom floor of Terrarium, you are hearing two stereo tracks further spatialized into three-dimensional space. To my surprise, it works amazingly well and adds such a strange verisimilitude.
