MPEG-I

MPEG-I is the new under-development standard for virtual and augmented reality applications. It aims at creating VR experiences that are natural, realistic and deliver an overall convincing experience – not only for the eyes, but also for the ears.

Imagine hearing a concert in VR where you are not rooted to just one spot, but can move freely around the concert hall – what a tremendous experience! Or imagine the broadcast of e-sports or sporting events in which users can move around the stadium while they watch the game. This is one goal of MPEG-I.

Previous solutions enable a visual or acoustic experience from one observation point in what are known as the three degrees of freedom (3DoF). By contrast, the upcoming MPEG-I standard will support a full six degrees of freedom (6DoF). With 3DoF, users can move their heads freely and receive input from multiple sides. But with 6DoF, the user is able to move within the virtual space. They can walk around, explore every viewing angle, and even interact with the virtual world. The situation is similar for augmented reality (AR), in which the user acts within the real world that has been extended by virtual elements. For example, you could arrange several virtual musicians within your living room and enjoy your own personal concert. 

All this requires sophisticated technology to produce a convincing and highly immersive audio experience, and it involves taking into account many aspects of acoustics. One example is sound propagation in rooms and around obstacles. Another is sound sources, which can be either static or in motion (the latter produces the Doppler effect). They should have realistic radiation patterns and size; for instance, if we want to auralize the sound of waves on the beach, the audio source extends to a kilometer or more. Currently, we are investigating the complex phenomenon of the diffraction of sound around obstacles or room corners and how this can be rendered efficiently in real time.

It will still be some time before all of this can be implemented successfully. For the standardization of 3DoF applications, we can draw on our MPEG-H expertise, but many other technical developments are necessary for truly immersive rendering of 6DoF sound. If all goes well, it should be available on end devices in the year 2025. Overall, we want to develop a long-term stable format for rich VR and AR content that can be viewed by users in 10 to 20 years just as easily as when it was first created – much in the same way that an mp3 file still plays seamlessly today, 30 years after the underlying standard was finalized. Reproduction using MPEG-I would be possible both with dedicated receiver devices and on everyday smartphones. As a result, it would be feasible to distribute VR and AR content as a next-generation video service over existing distribution channels. In this way, providers can offer users truly exciting and immersive experiences with entertainment, documentary, educational or sports content.

MPEG-I Immersive Audio - The Upcoming New Audio Standard for Virtual / Augmented Reality

Quality Testing for AR and VR in MPEG-I Immersive Audio

MPEG-I Immersive Audio — Reference Model For The Virtual/Augmented Reality Audio Standard