Em Halberstadt is a sound designer at A Shell in the Pit, where she works on audio for indie games. In addition to exhibiting an installation at the MOR, Em also works on the museum’s sound design. In this interview, she talks about her creative process, tools, and the difference between designing sound for traditional games and VR experiences.
Pronouns:
She/They
Based in:
Vancouver, Canada
Hobbies:
Camping, Roller Skating, Live Music Shows, Painting, Writing, Board Games with friends etc.
Fun Fact:
I can make a lot of different sounds with my voice. I can make the sound of machines or animals. I think I do a pretty good AI voice.
My background started when I was quite young, I was always making art. And I used to draw a lot. I would make a lot of home videos with our family camera. And that kind of led me to University to study in a programme called New Media. There, they taught a lot of installation art and a lot of physical.. and there was coding - which I did not like - and so I just kind of made everything into a visual and sound installation. And that kind of led me to sound. Then I came to Vancouver and have started working with ‘A Shell in the Pit’ for a few years now. And I guess I got into VR through that, with Gord who is working on Colin and Sarah Northway’s ‘Fantastic Contraption’. I don’t do a lot of art in VR, but I did for this. Mostly, I just do sound design for indie games. I’ve done sound on ‘Night in the Woods’ and ‘Wander Song’, ‘Ikenfell’, ‘Untitled Goose Game’ - and yes I work on Fantastic Contraption too.
Let’s see. Getting into VR for me was both deliberate and a happenstance thing for me, I think. In my program it was New Media. So they were teaching a lot along the same lines as VR, but they did not mention VR too much. It was a few years ago, I was always really interested in immersive art, so it kind of made sense to go into this direction. The piece that I made for the MoR - I actually made previously and in my thesis I kind of made a physical version of the ‘Memory Machine’ - just kind of made a big box that you could stand in and then I projected video onto all the walls. So it was kind of like a physical version of what is there in the MoR now. I guess that helped me prepare for it - especially with sound. Just thinking through the immersiveness of it.
I feel like throughout the time that I have been creating I have always had the same idea coming back to me - which is the idea of taking our memories and capturing them, on a machine, which we do on our phones all the time - but I wanted to explore the idea of having that in a museum. So I created this box that you would walk into - and I got one of those mirrors that you see on the top of, or inside of stores, like a spherical, half sphere mirror. I attached that to the end of the box and I projected into it, so it was kind of like a fish-eye type effect. And I had a 10 minute video playing on a loop - of just this person’s memories. It had been someone who worked on the machine, and they kind of degrade over time. So that was the first version.
What I put into the MOR - I created that in Tilt Brush. It was a similar kind of circular/spherical design, but there’s no visuals. It’s all done with audio. So you stick your head in and you will hear these recordings that I recorded with binaural microphones - which is a type of microphone that records how you hear as a human. With a normal microphone what gets recorded in not how we hear - as we have two ears and our head affects the frequency of the sound that we hear. So binaurals are microphones are basically that you put in your ears. Ever since I got them I have been recording random snippets of my life, things that I thought were interesting without really having a purpose. And I thought that this installation would be a really good time to use them. So, if you put your head into the sphere, you will hear..and feel sounds as if you’re in that space… because of the nature of the recordings.
In the center, there is a voice - which is me - and it’s like someone talking about how the memories are fading away and the idea of the intangibility of memories. How, as humans, we put too much effort into trying to capture them instead of just living.
My first VR experience was, I think, with ‘Fantastic Contraption’. It was quite an impact to feel the sense of being in another world and kind of understand all the possibilities that could bring you, and how immersed you get in the world. It took me some time before I could be in VR for a longer period of time, I would get kind of motion sick. But that is definitely better now.
Yeah I think it being the first one that I tried - and I spent a lot of time in Fantastic Contraption - listening. It was my boss - Gord McGladdery who did the sound. So that was my first experience of what sound could do in VR. In that game, it’s really musical and other worldly. So yeah, I think that would be the one with the most impact.
To edit, I use a program called Reaper. And I think it’s a really great tool as it’s open source and very flexible.
For the most part, when I’m working, I do a lot of recording. I don’t actually use a lot of tools in the digital sense - I don’t use a lot of plug-ins. The first thing I usually think about is - can I record this? And I’ll go out and walk around my house and see what objects can make an interesting sound. I would go outside and make noises with my voice, into the microphone. I think it allows me to be more creative in a sense, because I’m making my own source material as opposed to just finding stuff in a library, or using synthesis. Which I think is also great for a lot of reasons but this is kind of the way that I get the most enjoyment out of doing sound design. I used to have a little sound fort, which was like sound panels that were stacked up - and I would go in and it would make the room quieter.
Now I have just a microphone on my desk and I like that. It’s a lot easier to just grab it whenever and there’s less of a barrier between setting up a microphone in a different area, and I can just kind of go for it. So there are a lot more reflections a lot of the time, but I can still move it around if I need to. So I guess the most important tool that I have is the microphone. And with that there are very few limitations.
Unless there is a situation that I need a sound like that of a European bird or something. In which case I would have to use a library! I think most of my sound design I can do with a microphone.
A lot of the work I do is with found objects. So instead of trying to think about it rationally - that what could I use to make this sound? - I like to take inspiration from what’s around me. So going on walks, seeing what I can do with my voice is usually really helpful.
I think that generally the trajectory of most technology is that it is getting less clunky and more accessible - which is really great. I think there used to be a lot of pressure to have the best equipment, kind of in any medium. But sound design as well.
Now you can kind of get a low to mid end recorder, and you don’t have to worry about quality as everything is getting better. I often tell sound designers that are starting out and wondering what equipment they should get - what microphones… I tell them that it is more important to just record a lot and practice, and not really worry about having the best equipment.
My creative process usually starts with looking around in my environment and seeing what I can use. Recently I was making the ‘teleport’ sounds for the MoR, and I knew that I was gonna use stone. So I did spend some time listening to sounds in the library to see what would work. But then I knew that I wanted some really clean sounds [that were also] kind of, almost musical. So I went onto my patio and I got a big rock and a microphone. And I recorded a few impacts of rock on brick. I find that doing that - it does take a lot of work to go outside and set things up to do the actual recording - but it saves a lot of time in the long run. I don’t have to look through a lot of sounds and I have a limited palette that I can work with. And that makes me do interesting things with it, when I limit myself.
I guess the process can be surprising if I go through a bunch of source materials and there’s a sound that I wasn’t expecting to use, but it just sounds right. Lots of times when you’re designing sound, you think of the object as the sound. Instead of just turning that part off and just listening to what the emotion the sound gives you. Or how… something I often think about when I work is how ‘pleasant’ does it sound.
If I am recording a door, and putting the sound of a door - in some where, I could have the door recorded and think okay that’s a door, it should work. But then it doesn’t actually sound like a door. Maybe there’s a scratchy, high end frequency that is uncomfortable to listen to. So yeah, listening as if you don’t know what it is can yield some pretty cool results. It’s kind of the same as when you’re drawing and you wanna draw a door (laughs) rather than just drawing what you know, you draw what you see.
Well I guess, one of the examples of an object where it’s… the sound of something that isn’t what it is, is... I think when I started out, I researched a lot of these - looking into what other people use. And one of them is - I think it’s baking soda or flour for snow. It happens a lot, it’s often not the actual thing you see.
Especially with stuff that doesn’t exist. Like with creature sound design people often use a pig screeching - that is a very common one. And camels sound like monsters.
With sound design, you combine layers. So if I’m doing a monster or a creature I’ll often just do it myself, first. And often, if you slow down a human sound, it can sound like a monster. And then layering different animals, also slowed down animals, on top of that usually works. A camel I think is my favourite. Weird sounding animal. A camel and I think seals. Yes.
SFX reel for Vancouver-based A Shell in the Pit, where Halberstadt works on audio
I mean I guess traditionally game audio has been pretty—people have used a lot of synthesis and it has been abstract. But in VR, a lot of it has to be pretty real sounding as people kind of expect to be in that environment. And, for instance, there’s usually a lot of physical objects that you can throw around and that has to sound realistic. If you throw it really hard against the wall, it has to make a bigger impact and then when it hits the ground, it will sound quieter… those exist in normal games as well. But it has to be a higher level of detail in VR. Same with the environment sounds - like the ambience. For it to be immersive it has to be all around you. So, traditionally you would just make a stereo ambience - which is left and right with speakers. And VR has what we call a quad ambience, which is all around you… a point in your front and behind you, which calls for a lot more detail.
Same with putting sounds on any object you can think of in a space. There’s a world that I did ambience for recently in the MoR that has many different… there were pipes. So you put the sound of hissing pipe on it, and when you get closer, you hear it louder… kind of thing. I think it also encourages people to explore more around them, instead of just standing and looking at things. If they hear a shift in their environment, they’d be more likely to explore.
I think part of what drew me towards being a sound designer was that the idea of promoting listening. I think that it’s often the case that people don’t really pay attention to the sounds around them. And I think that’s a little dangerous because even if you’re not paying attention, these sounds do affect you. So if you’re constantly immersed in the city you hear all this construction and all these cars. It’s gonna subconsciously stress you out. And you might not really know why, you might just think it’s something else. But if you… a lot of people, when I talk about this, kind of get mad at me (laughs). That, because of… once you start listening - it just grows exponentially. And you’ll begin to really notice things, making it tough to turn some things off. But I think that the benefits outweigh that as you learn that if sound is stressing you out, you will give yourself more opportunities for silence. And you’d even appreciate silence. By going into the forest, or by just listening to the more soothing and natural sounds. So I think that using sound for meditation is a really good way to solve for - if you find it hard to meditate. I don’t do the super conscious meditation myself, but I often will just sit and listen. And it’s kind of a different form of it for me. If you are sitting and listening to the sounds around you in a forest or something as things are happening - it really situates you in that moment. As, to pay attention to a sound means that you have to be listening at the same time that the sound is happening.
So, currently I am working on a few different indie games. I am soon to release a game called Untitled Goose Game. And that is a game where you play as a goose and then just go around bothering people. I’m also working on another game about creativity and painting or drawing called Chicory. And another game about a school of magic called Ikenfell. So we (A Shell in the Pit) is actually a team of 6 people and we work on projects together and also have our own. So I am working on a few projects with the team.
If you’re a VR artist and you wanna use sound or experiment with sound more, I think that the best thing to do would be to get a little recorder. You don’t even need to get a nice microphone to go with it. Just a cheap one should do. Just start recording anything you can think of and then you can make yourself a little library to work with. Specifically if you’re making something that you want sound for - you could try just using that recorder and record your voice. There are a few games that use voice for all the sound effects. So there’s certainly a lot you can do with your voice. If you’re just starting out, play around with a program called Audacity. And Reaper - the program I use - it kind of seems like it might be complicated. But if you put in a few hours into learning it, and if you know editing software already- like video editing software, then it is quite similar to that.
If you’re recording without knowing too much about it, I guess I would say - always use headphones. Yeah, just make sure you’re listening to what you’re recording.
—
This interview is part of the MOR Artist Spotlight series, which features creators using VR as an artistic tool or medium of expression. You can also view a spatial version of this interview in the Museum of Other Realities.
