“We will all be using generative AI”

AI algorithms that generate text, images, or videos based solely on prompts – i.e., input in human language – will play a major role in business, research, and society in the future. Known as generative algorithms, these broaden the options for making use of artificial intelligence. And Fraunhofer IIS is at the forefront of their development.

Ever since the release of ChatGPT, generative artificial intelligence has been inspiring people all over the world. However, the technology’s benefits go far beyond text-based chatbots. In the future, generative AI will also help with automatic announcements on trains or in industry, using natural-sounding spoken language. The Audio and Media Technologies division at Fraunhofer IIS is driving this form of generative AI forward in a number of projects.

It happens every day in open-plan offices the world over: On screen, you can see your colleagues discussing an important topic. But in their background, other employees are having their own meetings, so you end up hearing rather more of the background noise than you do of the actual topic you’re trying to discuss. In the future, though, once generative artificial intelligence finds its way into laptops, smartphones, and the like, this scene will be a thing of the past. Conversations happening simultaneously in the background will be almost entirely filtered out, thanks to upHear Target Speaker Extraction. To date, this has been achieved using traditional, or discriminative, AI: all the model needs to be able to generate a digital fingerprint of someone’s voice is a few seconds of training data. The fingerprint is used to amplify the speaker’s voice and block out background conversations. “Thanks to AI methods, this already works very well,” says Jan Plogsties, Strategy Manager Generative AI at Fraunhofer IIS. The institute developed the technology as part of its many years of work on solutions to improve audio quality. Indeed, various products in the Fraunhofer IIS upHear family make use of AI – from smart speakers and smartphones to microphones for conference calls. Generative AI will make operating the technology even more efficient in the future. For instance, it could optimize the quality of the spoken word, even in the presence of extremely loud noise interference such as ventilation, vacuum cleaners, or street noise.

Generative AI differs from discriminative AI in that it can generate completely new content that has never existed before. That includes not only text but also new images, videos, and audio content. Having seen huge amounts of data during their training, the models can generate very plausible new content on the basis of very little information. This is the decisive advantage over traditional AI algorithms.

Generative AI is set to become more important

Target Speaker Extraction is just one example of how artificial intelligence could come to be applied in more and more areas of private and public life in the future. The introduction of ChatGPT has already shown just how much potential lies here to be tapped. Plogsties is certain that generative AI is set to become even more important: “We’ll all be users of generative AI,” he says. The Fraunhofer-Gesellschaft has recently set out a clear position on research into generative AI. Its position paper states that through its institutes, it is involved in pioneering projects in the field of generative AI. The paper then refers to the OpenGPT-X project, in which Fraunhofer IIS and Fraunhofer IAIS are working on powerful open large AI language models for European languages. OpenGPT-X is specifically designed to meet the needs of European companies while being trustworthy, versatile, and reliable. “Our experience with OpenGPT-X has been very good,” Plogsties says. Nevertheless, language models are merely one of the institute’s many points of contact with generative AI. “We have over 30 projects at Fraunhofer IIS that are fundamentally applying generative AI,” he explains. Not to mention all the work that is already being simplified through the use of generative AI tools.

Germany’s three areas for action

Both the Fraunhofer-Gesellschaft and Plogsties himself identify three key areas for action in the field of generative algorithms. “We have to master this technology stack in Germany,” Plogsties says. This is the only way to guarantee that Germany will continue to play a pioneering role in development in the future. Added to this is the computing capacity, which needs to be built up much more, and the data itself. “Data quality has a huge part to play,” Plogsties says. In contrast to ChatGPT, which is based on the texts available on the Internet and is therefore prone to errors, generative AI for scientific use must deliver reliable results. To this end, Fraunhofer IIS is already in talks with companies that provide their high-quality data for training generative AI models.

At present, the trend in generative AI is toward local models. “We’re currently working on bringing the quality of large cloud models directly to devices,” Plogsties says. This offers many benefits, among them energy efficiency and sustainability – areas in which artificial intelligence often comes in for criticism. Smaller local AI models also require comparatively less power. As Plogsties sees it, generative AI has so far shown only a fraction of what it’s capable of. “And Fraunhofer IIS is playing a key role in shaping this development,” he says.

Article by Julian Hörndlein, freelance journalist and PR copywriter