Over the last year, the world has become captivated by advances in virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies. For the unacquainted, virtual reality can produce computer-generated, three-dimensional environments which can be interacted with in a deeply immersive way.
Augmented reality and mixed reality are the blending of virtual reality with real life, as developers can create imagery to blend in with features of the real world. For example, this technology has been used in museums to enable visitors to perceive an additional layer of information by animating the subject matter—such as displaying the missing parts of a damaged statue to replicate what it may have looked like in its original state.
While all three techniques are incredibly powerful tools, optimization of this technology relies on dynamic, binaural 3D sound.
Binaural 3D sound is much more authentic to our ears than two-dimensional stereo. It is designed to replicate the way we hear spatially, leveraging how humans consume auditory information in our natural environment. 3D sound engages the listener by offering a spatial bearing that enables them to sense where they are relative to the noises around them. In a 3D soundscape, the origins of sounds can perceivably move about the listener, locating the listener as if they were standing in the recorded environment.
Metaphorically speaking, VR, AR, and MR are still in their “Charlie Chaplin phase” because they are, in essence, modern-day silent films. While wildly entertaining for our eyes, it’s not until these nascent visual technologies are successfully integrated with a new generation of binaural 3D sound that they will offer a truly immersive experience.
By inserting 3D sound, new spatial information is introduced to the virtual experience, enabling audiences to sense things happening behind them, or elsewhere in their virtual environment, completely independent of their eyes. Just imagine yourself in a scene from a horror movie where you’re being chased through the woods in the dark, and you can hear your assailant in hot pursuit, the crunch of the forest floor under your feet, and the snapping of small limbs coming from behind you. As you turn your head to look over your shoulder, sounds shift slightly, compensating for the re-positioning of your ears. Your assailant sounds closer than ever and your stumbling feet kick up a storm of dead leaves, breaking the silence of the dense woods. Manipulating this type of audible sensory perception has the potential to completely reshape the visual virtual experience.
While the marriage of 3D sound and virtual reality may sound simple, it’s actually unchartered territory—and it raises many questions for storytellers looking to utilize this new medium. How are audiences expected to behave during this type of immersive experience? Are storytellers expected to find a role for their audience to play—giving them a voice, as it were? How can virtual reality truly complement live subject-matter without stealing the show? And should a virtual narrative follow a single path—or should the audience be at liberty to choose their own narratives and adventure? There’s currently no blueprint for piecing together the storytelling that’s best suited for this type of virtual medium. As far as content goes, there is consensus about only one thing: in a virtual world, it’s all about “storydoing,” not storytelling.
The VR, AR, and MR industry is just now beginning to explore the possibilities of binaural 3D audio, and efforts to harness the technology are still very much in early development. However, we expect it to advance quickly.
In less than five years, 3D spatial audio is expected to revolutionize our standard for multimedia listening. Similar to how high-definition television has enhanced the everyday viewing experience, binaural 3D sound is expected to reshape our listening experience and redefine the production of music, movies, radio, and television programming—and yes, VR, AR, and MR content as well.
The future of VR, AR, and MR is on an inevitable collision course with binaural 3D sound and the pending explosion of creativity could be amazing. Keep your eyes open, but be listening for it. It’s coming soon to a pair of ears near you.
David Falter is the president and CEO of Antenna International. He is a six-time CEO and board director across numerous industries—including electronic travel distribution, business services, systems integration and software development, and cloud-based storage.
