Fraunhofer IIS Audio and Media Technologies at CES 2021

xHE-AAC

xHE-AAC is the latest member of the MPEG AAC audio codec family. It is natively supported on the latest Apple, Android and Amazon products, and Fraunhofer’s xHE-AAC implementation has been licensed to Microsoft. The codec enables consumers to enjoy uninterrupted streaming of all types of content – such as movies, music, audiobooks or podcasts – with stereo services at bit rates ranging from 12 kbit/s to 500 kbit/s and the ability to switch between them seamlessly. The mandatory MPEG-D DRC metadata provides loudness and dynamic range control so that xHE-AAC can play content at a consistent volume and deliver the best possible user experience in any listening environment, on any device.

Try the interactive xHE-AAC demo here and listen to the benefits that xHE-AAC offers over conventional HE-AAC.

Fraunhofer IIS offers a new web-based test service that developers and manufacturers can use to validate their implementations of the xHE-AAC audio codec for compliance with MPEG standards. The service, which is available exclusively at https://test.xhe-aac.com, is free to use upon registration with Fraunhofer and will test both encoders and decoders. The test service provides an easy way to further validate implementations of xHE-AAC for all the advanced features, as well as testing general coding tools. Fraunhofer will be licensing the use of its “xHE-AAC” trademark on a no-charge basis for use with products that successfully pass the test service tests. The xHE-AAC trademark program will extend our trademark programs from MPEG-H into the AAC codec family.

• New xHE-AAC web page demonstrates dramatic audio quality improvements and user benefits
• Fraunhofer IIS licenses xHE-AAC audio codec software to Microsoft

MPEG-H Audio

The MPEG-H Audio system delivers enveloping immersive sound and allows consumers to choose between different audio presets or to adjust the dialogue volume. Regardless of the device, the MPEG-H Audio system delivers the best sound experience possible - in the home theatre as well as on smartphones, tablets and Virtual Reality devices.

Find all you need to know about MPEG-H Audio here.

LC3 / LC3plus

The new LC3 / LC3plus audio codec was developed in order to solve essential shortcomings present in today’s wireless communication platforms such as Bluetooth and Digital Enhanced Cordless Telecommunications (DECT). The codec’s operation modes range from medium bit rates for optimal voice transmission up to high bit rates for high-resolution music streaming services. At the same time, the codec operates at low latency, low computational complexity and low memory footprint.

As the quality of streaming content increases, so does the need for standardized solutions in audio coding. The complexity of the LC3plus codec meets the requirements of wireless communication platforms while still operating at low latency and low memory footprint. Even at low data rates, LC3plus provides high-quality speech and audio transmission. The LC3plus bit rate is roughly half that of legacy codecs, which facilitates low-energy audio services – making it possible to extend the battery life of products and create smaller devices. LC3plus features greater robustness against transmission errors, even lower encoding delay, and the ability to play back audio in high-resolution quality, for example on wireless headphones.

We designed a browser-based interactive demo, showing the audio benefits of LC3 and LC3plus. Get in touch with us via email to schedule your personal demonstration slot. 

• Fraunhofer IIS licenses LC3 audio codec software to MediaTek
• Fraunhofer IIS licenses LC3 audio codec software to Broadcom

Enhanced Speech Recognition in Smart Assistant Devices

Fraunhofer upHear Voice Quality Enhancement is a smart-assistant-ecosystem agnostic microphone processing technology. The software is designed to facilitate voice-controlled human-machine interactions using microphones built into mobile phones and smart assistant devices such as smart speakers or smart soundbars. It allows the smart assistant to understand far-field voice commands and enables barge-in by removing interfering sounds captured by the device’s microphones, extracting the user’s voice and cancelling out acoustical echoes that would otherwise make it impossible for the HMI to understand the user’s request.

Fraunhofer upHear Voice Quality Enhancement is a fully integrated and flexible solution for a wide range of mobile and smart assistant devices, as well as conferencing solutions. The technology combines advanced source localization and beamforming techniques with echo and noise reduction algorithms, thus providing outstanding voice quality even under unfavorable acoustic conditions. Advanced multichannel acoustic echo cancellation allows for barge-in functionality in an always-listening operation of the voice-controlled HMI.

Even though the technology supports single-microphone use cases, we recommend the use of microphone arrays to further improve the user experience in challenging conditions, especially for far-field applications.

Try the interactive upHear VQE demo here and reach out to our experts via email to schedule your personal demonstration slot. 

JPEG XS - the new low complexity codec standard for professional video production

The standardized JPEG XS codec was developed to handle media workflows from acquisition to distribution by using Ethernet settings and infrastructure only. Until very recently, digital image transmission for production and contribution could be done only by using specific interfaces such as SDI, IEEE1394, or CameraLink. However, with the availability of higher bandwidth of Ethernet interfaces, the handling of highest-quality images over internet protocol (IP) in local and wide area networks was required and JPEG XS is a codec enabling these requirements. 

An update to the new video production codec for professional video

A low compression of up to 10:1 allows near-transparent transmission. JPEG XS – developed to offer lowest l atency for multiple encoding-decoding cycles and moderate computational resource requirements while preserving image quality at the highest level – fulfills these demands to facilitate production/ contribution settings, even for 4 and 8k images. The core coding system of JPEG XS was standardized in ISO at the end of 2018 as ISO/IEC 21122-1, the remaining parts in 2019. What is available for industry applications today are the compression of RGB and YCbCr images in 444 and 422 sampling formats with up to 12 bits per component for broadcast and prosumer use cases. Some smaller extensions, like compression of 420 sampling formats and lossless compression, are under development. 

Integration of JPEG XS into cameras and image sensors

The current standardization activity is a big step forward to enable JPEG-XS for compression of RAW Bayer image data. During this JPEG XS development phase, a PSNR gain of 5 dB in coding efficiency could be achieved and will be included in a new amendment. This allows the industry to integrate the codec into today’s cameras and image sensors. It offers the use of the codec in the complete production pipeline – from the image capturing to the distribution encoder. It facilitates the use of the codec in other use cases, like integration in cameras for machine vision, automotive, or high quality surveillance, too. JPEG XS already exists as transport and file formats, like RTP, MPEG2-TS, JXS, MP4, and HEIF. The standardization of JPEG XS inside the MXF file container is under progress in SMPTE under the item ST 2124. With these activities, a complete suite of formats is now available for JPEG XS allowing the transport and storage of this format in the postproduction workflow.

JPEG XS SDK available

Fraunhofer IIS offers development kits for CPU and GPU usage, as well as consulting projects for integration into products to the industry. Initial implementations for JPEG XS were carried out successfully, even in 8k.