Futuresound: Will Accuracy Overcome Familiarity For VR Audio?

2017.06.29ㆍ by Gaudio Lab

Futuresound: Will Accuracy Overcome Familiarity For VR Audio?

John Dewey, who is known for exploring the philosophy of pragmatism, proposed that experience is based on two principles. The first concept of continuity explains that all experiences affect future experiences, for better or for worse. Experiences, therefore, are not independent of each other nor are they one-time events — each one has a relationship to something that happened in the past or will affect something in the future. For example, the fact that you are finally watching a VR piece is related to the fact that you recently got a membership for a VR content subscription service like Wevr’s Transport VR. Or maybe your enjoyable experience of The Mummy VR Zero Gravity Stunt Experience inspired you to go watch The Mummy movie in theaters.

Within this continuity framework, the VR listening experience has the potential to be very confusing. The way you hear sounds in VR might be totally different from the way you have been listening in the real world or while consuming more traditional content. At a concert or a conference, you see where the sound is originally generated from — the musician or the lecturer. However, unless you’re a VIP every time, you are actually hearing the sound from loudspeaker locations and not from the actual sound sources. This discrepancy also occurs when you watch something at the theater. The screen, filled with things that are responsible for the sound, is placed at the front while the sound is actually projected from speakers in various locations. Fundamentally, however, people are familiar with this listening experience — visuals are in front of them and sounds don’t necessarily match what they’re seeing.

Discrepancy in sound perception

We build new knowledge through experiences and all of our current knowledge is based on the experiences that came before. Even our imagination, a realm that is not physically accessible in the real world, is influenced by previous experiences. The more experiences we have, the more creative we can try to be.

Consuming different styles and concepts of VR content will be the key to building our experience base moving forward. We might learn that positioning ambient sound to the sides is more comfortable than spreading it throughout the entire scene. We might learn that volume manipulation could be effectively used to help people adjust more easily to this relatively strange listening experience. Or the decision might still fall down to familiar but false or strange but true, and even then, only after thousands and thousands of experiences have been accumulated.

3D Audio is the Hottest Opportunity in VR, Here’s Why

3D Audio is the Hottest Opportunity in VR, Here’s Why Increased Consumer Interest and A Higher Standard for Immersion When people watch content via a traditional medium like a TV or a theater, the screen in front of them represents a virtual world. There’s a separation between the real and virtual world, which makes it difficult for some viewers to remain engaged with the content. Through VR, however, people can isolate themselves completely from the real world, and when that happens, the entire adventure becomes incredibly visceral. “Being there” is the whole allure of VR, so sounds must be 3D to support the immersive nature of the experience. Sound spatialization consists of placing positional data to sound sources and gives the listener the impression of a sound source within a 3D environment. When the team at Gaudio Lab, a leading spatial audio company, showcases two different demos — one with spatialized audio and one without — they notice a common pattern of reactions. After taking off the headset, people profess “The spatialized one felt so real” or “I didn’t know that audio made such a big difference.” The hunger for spatial audio becomes insatiable once consumers experience it at least once. There’s no going back. Powerful New Tools and Revenue Streams for Content Creators One exciting opportunity for 3D audio centers around live concerts and sporting events. With VR, content creators can overcome physical limitations to serve the audience by reproducing and redistributing live events. But the true seduction of VR is the power to create experiences that could never exist in the real world. Why sit in the nosebleeds of a concert when you can be on-stage in VR, inches away from your favorite artist? As more fans engage with performances that way, it’s not crazy to think that 3D audio could lead to entirely new musical styles, potentially disrupting the music industry. When music is delivered in stereo formats, the sonic experience is shackled by the finite quantity of loudspeaker placement. 3D audio can be delivered via an unlimited number of virtual speakers if the music sources are in an object-based format, or structured as individual mono signals. This allows content creators to experiment with new methods of sound design and produce projects that are totally different from the typical music consumption experience. Music is about to leap past traditional techniques, and performances can now take place all around you, not just in front of you. Binaural Rendering Requires Talent and Technology The combination of consumer interest and creator need has given birth to a thriving 3D audio market with high demand, but there are plenty of challenges when it comes to delivering the technology on a practical level. A personal and mobile VR experience inherently requires a pair of headphones for audio output, and delivering 3D audio through that two-channel format instead of layers of loudspeakers is no easy task. That’s why binaural rendering plays such a critical role in the VR listening experience. This signal processing technology synthesizes 3D sound scenes comprised of object, channel and/or Ambisonics into two-channel outputs.The corresponding signals should then be rendered in a way that reflects where the sound is actually coming from, taking into account the relative direction and distance between the sound source and the listener.

2017.05.02

The Future of VR Audio – 3 Trends to Track This Year

The Future of VR Audio – 3 Trends to Track This Year 2017 has pushed the VR industry forward in countless ways, including the recognition of audio as an absolutely critical element in VR experiences. Here are a few trending efforts that creators are using to push the envelope with sound. The industry will embrace object-based audio for every kind of experience. Utilizing object-based audio also gives more creative freedom to content creators since it’s easier to manipulate post-production effects on a single sound — think of it as a single raw element as opposed to a big, messy sound glob. In addition, object-based audio works perfectly for 6DOF (six-degrees-of-freedom) VR content, which is rapidly growing in popularity. 6DOF content is just like a game — the character moves around within the space in every direction and has the agency to interact with objects in the environment. When the character does either of these things, the sound needs to change accordingly. Because it is better at pinpointing sound and easily reflecting the changes during gameplay, object-based audio has actually already been used in 3D game engines for quite some time. As more 6DOF content is being built on game engines, it’s plausible that more audio engineers will be forced to learn how to mix and master sound in game engines rather than their traditional Digital Audio Workstations. Quality VR content will be published with more players embracing spatial audio. These limits on publishing platforms have discouraged content creators from fully embracing spatial audio in their productions. Still, as sound is met with increased appreciation, renderers or players will eventually have to support spatial audio. When Vimeo launched Vimeo 360 in March to support 360 content, a huge amount of the requests from users involved a desire for a spatial audio feature and the official help page states that they are planning to support spatial audio in the near future. Smaller players and platforms will follow the path laid out by Facebook, YouTube, and Vimeo. As user standards for VR content quality continue to rise, adoption for spatial audio will race to keep pace. Out of the many reasons that have kept content publishing platforms from adopting spatial audio, the primary one has been the absence of a dedicated VR audio format along with a compatible renderer. With more emphasis being put on object-based audio, Ambisonics alone will be phased out as the standard format of the future. Creators will push beyond post-production to create new listening experiences. Sound is not just a storytelling cue that can be used to encourage VR users to look in a certain direction. In some new use cases, you are actually able to hear certain sounds over others within the same experience if you want. The following recorded 360 video, for example, lets users hear what they are looking at more clearly than the other instruments placed all around them. This new wave of sound won’t just be part of an evolution of current techniques. In many cases, it will give way to revolutionary new forms of entertainment. The virtual canvas for artists is expanded 360 degrees horizontally and 360 degrees vertically beyond the physical dimensions of a stage in real life. Musicians will now be able to play with “virtual location,” along with their traditional considerations of pitch, loudness, and timing. They’re also learning how to exploit human auditory perception to influence these experiences at an even deeper level. Some psychoacoustic principles that you have already experienced in real life can be taken advantage of in VR to make each experience different at the individual level. While there is a whole lot to consider in that realm, our collective knowledge about it continues to grow.

2018.07.05