“It will have a massive impact on the economy and society”

Professor Heuberger, cognitive sensor technology is one of the Institute’s two main themes, along with audio and media technologies. Why did the Institute decide to make these two research areas its main themes?

The choice of audio and media technologies was quite straightforward. This is a very high-profile research area, and one where we are global leaders in technology. That is why we will continue to drive it forward on a strategic level and have made it one of our main themes.

For our second theme, we focused on identifying what connects all the research that we do in different areas – and it is the development and use of sensor technology that is becoming increasingly cognitive. In “cognitive sensor technologies”, we have come up with a handy term that draws together the different strands of our work. “Cognitive” in this context means that the sensors don’t just capture measurements and pre-process them using conventional signal processing methods, but are also capable of drawing conclusions from those values.

Are you saying that the sensors think?

That is a bit of a stretch. However, they are equipped with additional capabilities based on machine learning processes. That enables the sensors to build up empirical knowledge from their environment, so that they can work out particular patterns and trends from the signals.

So the thing that is new about this kind of sensor technology is that the sensors learn independently?

Yes. Because we are now also using processes based on machine learning in our work on sensor technology and data transmission, we can do a lot more analysis. The findings from that can form the basis for possible business logics. There is very little more we can achieve using the traditional suite of signal processing technologies. The additional technology of machine learning can open up a whole new range of possibilities. That work is already well underway.

Do those new possibilities relate primarily to industry and the economy? Or will people come into contact with them more and more in their everyday lives, too?

Definitely! One area where there will be lots of applications for this kind of sensor technology is the car. We see that very clearly, of course, in the development of autonomous driving capabilities. But this sensor technology can play a role in other areas, too – by detecting the driver’s level of fatigue, for example. It can even go beyond that, to detecting the driver’s state of mind. The car can generate autonomous responses to that information. At the most basic level, that may involve changing channels on the radio, but it may also mean adapting the chassis dynamics or the sensor technology to the environment – using radar sensor technology, for instance.

One of the main application areas for cognitive sensor technology: mobility

"If a foundry, say, deploys systems of this kind, it can improve product quality significantly."

Your partners and clients are business and the public sector. What are the specific application areas at which your research and development in cognitive sensor technology is targeted?

At the moment we are focusing mainly on five application areas: mobility, communication in the whole sphere of IoT, the human senses, material testing and monitoring, and supply chains.

Can you give us a concrete, practical example from one of those areas where cognitive sensory technology is in use, and say what improvements it has brought with it?

It is being used to monitor materials in components, for example. That involves having X-ray sensors take 3D pictures of components or parts, and using the image data to assess how the manufacturing process is going and what the production quality is like, for example. This requires the use of machine learning-based approaches, to ensure automatic monitoring. If a foundry, say, deploys systems of this kind, it can improve product quality significantly.

So these technological solutions don’t just work for big companies – they are also designed for use by small and medium-sized enterprises?

Absolutely! Our job is to build up a critical mass of research expertise and to make it available to business customers. That is particularly relevant to small- and medium-sized enterprises. Often these SMEs require only single applications, so it is hardly worth them developing their own research capacity. We can help them out there. It also part of our remit to produce individual technological components, which we can then offer to SMEs under license.

"This research will play a major role in shaping the future development of technology."

What existing components are you working on in the Institute as part of your research and development in this area? Which do you want to develop further?

Many machine learning-based techniques are in use even today, not just along our pre-existing innovation chains in the areas of circuits and sensors, communication technologies, and analytics and evaluation, but also in the development of services and business models. A large number of researchers – around 70 – have been working for a long time with techniques that are needed for cognitive sensor technologies. We now need to work out what kind of machine learning-based techniques, technologies and processes we need for our future business. We want to target investment on developing these methods and algorithms.

Is the Fraunhofer IIS doing research into cognitive sensor technologies on its own, or are you also involved in partnerships?

Exchange is the life-blood of research. In the case of cognitive sensor technologies, we are talking about relatively young discipline that has taken on much more importance in recent years. For that reason, we benefit from having a close relationship with our strategic partner, the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). However, we also collaborate with other universities such as LMU Munich, or the universities in Bamberg, Bayreuth, Passau and other places – both here in Germany and internationally. We rely on having good connections, as otherwise we wouldn’t be able to bring in the experts and talents who are helping us to develop the subject effectively.

How does this research benefit the economy and society?

This research will play a major role in shaping the future development of technology. Along with other major research trends like biologization and energy supply, equipping machines with cognitive abilities will have a massive impact on the economy and society. Our competitiveness is dependent on whether we in Germany can remain a technological player in this area. So this is not just an attractive research project for us, but something we actually need to get involved with. We want to exploit the potential of these technologies and to play an active part in shaping the future. If we don’t do it, others will …

Let’s return specifically to the work that you are doing. Looking to the future, what plans do you still have in this area?

We want to open up new business and new opportunities for all our operating areas based on cognitive sensor technology. What is more, we have already launched some new research initiatives. With our partner, the FAU in Erlangen, we are currently planning the ADA Center, which will provide a focus for machine learning-based research in the Nuremberg metropolitan region.

I am particularly excited about another new initiative, the Campus of the Senses, as it is about extending the digitalization of human sensory perception beyond sight and hearing. The fact that human sensory perception in all its complexity can be simulated and reproduced in a machine is impressive in itself. Together with our partners at the FAU and the Fraunhofer IVV, we will be able to research and digitalize the basic physiological and psychological mechanisms that govern taste in humans, and to transpose them to useful applications for society and the economy.

In the next five years, we want to generate a significant proportion of our income from cognitive sensor technologies.