“KI-PREDICT” – Intelligent process monitoring with on-sensor signal preprocessing

Group manager Dr. Matthias Völker and senior engineer Dr. Markus Stahl-Offergeld both work in the Smart Sensing and Electronics Division at Fraunhofer IIS. They are currently working on “KI-PREDICT”, a project funded by Germany’s Federal Ministry of Education and Research (BMBF) devoted to on-sensor data processing supported by AI functions. For this project, the two experts are focusing on the development of a “sensor interface ASIC” with integrated AI functions. One of the partners in the “KI-PREDICT” project is Rolf Slatter, the managing director of Sensitec GmbH since 2009. Sensitec is a global market leader in high-quality, innovative magnetic sensor solutions. We asked Matthias Völker, Markus Stahl-Offergeld, and Rolf Slatter to tell us about the project and their collaboration.

How did the KI-PREDICT project come about and what is the idea behind it?

Dr. Völker:The main focus of our department is on developing integrated circuits for sensor-based analog and digital signal processing. Thanks to developments in the field, AI functions can now be integrated directly in the sensor, thereby requiring very little energy. So rather than merely having an analog sensor interface and digital filtering, the KI-PREDICT project goes a step further, incorporating certain AI functions into the sensor itself. The sensor is designed in such a way as to allow data to be acquired, processed, and evaluated locally.

What is the goal of the project?

Dr. Völker: IIn this project we are developing an integrated circuit comprising both the analog interface, i.e., the direct connection to the sensor, and digital assessment capabilities with AI functions. These capabilities are then tailored to the applications envisaged in the project. This application-specific integrated circuit means we can process data faster than would be possible with a standard microcontroller. As a result, conventional industrial buses, which connect the sensor to the control unit, can be used. These buses only allow relatively low data rates. They don’t just transmit the measurement data, but also additional information obtained by means of AI, such as information about the condition of the sensor or the workpiece. In this way, conventional industrial buses are expanded with new functions. This expansion and integration are the goal of the project.

What new findings/expertise will Fraunhofer IIS gain from this project?

Dr. Völker: In this project we have the opportunity to develop accelerator units for AI functions and explore the most efficient way to incorporate these functions into our integrated circuit (IC). This process will unfold in collaboration with our partners, allowing us to test the individual functions and explore the best way to incorporate them. Accordingly, we can base our predevelopment work on the actual needs of the industry and resolve potential problems in advance before presenting the development to industrial customers. In addition, with AI developments it is always important to have enough data to work with. And this is precisely what the project gives us: It includes three measurement campaigns that will take place in the production environments of the participating companies. This will give us and our project partners extremely high-resolution real-time data that we can use to train the AI and test our developments in an applied setting.

How does the project relate to the Institute’s AI expertise? Are there any points of contact with other departments or divisions, for instance?

Dr. Völker: The project is an excellent fit for the Institute’s expertise in the field of AI. We are developing hardware components for AI functions that complement certain aspects of neuromorphic hardware, for instance. Our feature extraction can be implemented upstream of neural networks or used for classic machine learning functions. Meanwhile, we are also using these findings in collaboration with the “Machine Learning & Validation” group, which develops pure software products. With the results obtained from the hardware acceleration, we are in a position to create a new product in the project in this thematic area. This will give our special hardware an additional advantage over the standard product.

What does the development process for a sensor interface ASIC of this sort look like, and what added value does it provide?

Dr. Stahl-Offergeld: For every sensor interface ASIC, the first thing we need to do is understand what the customer really needs. The initial specifications are often based on existing discrete solutions and marketing goals. The challenge lies in understanding the overall system and designing a solution that satisfies the customer’s wishes while exploiting the benefits of CMOS integration, such as reduced power consumption. In general, the benefit of an ASIC is that the circuit is tailored to the application, whereas discrete solutions seek to meet the specifications using existing components. Its benefits include a smaller assembly space, lower power consumption, and higher performance.

What expertise within Fraunhofer IIS can you draw on for the development process?

Dr. Stahl-Offergeld: The Integrated Sensor Systems department has already developed a number of sensor interface ASICs, and you can find an experienced expert for any given specification at the division level at the latest. What I really appreciate here is the working atmosphere. Even when colleagues are busy, they will always take the time to discuss problems and issues.

What challenges do you face in the development process?

Dr. Stahl-Offergeld: The biggest challenge in the KI-PREDICT project is the complexity of the ASIC. It requires the development of new components both on the analog side, such as an analog-digital converter and an amplifier, and on the digital side, such as a RISC-V microcontroller complete with hardware acceleration for machine learning. As the controller is located on the chip itself, it has to be equipped with special hardware acceleration blocks and perform the feature extraction needed for machine learning. The goal is for the chip itself to analyze an entire measurement cycle, rather than record all of the values measured by the sensor and simply forward them. When a piece is pressed or formed, for example, the entire cycle has a duration of around 20 seconds to a minute. The chip analyses the measurement cycle in real time, calculates the parameters needed for condition monitoring purposes, and then conveys them to the higher-level system where the condition monitoring is performed. This decentralized feature extraction at the sensor, reducing the volume of data to be transmitted, makes it easier to integrate the condition monitoring process into an existing production system.

What has collaboration with your project partners during the coronavirus pandemic been like?

Dr. Stahl-Offergeld: It has worked surprisingly well. At first I was skeptical, as the project began during the lockdown, so there was no kick-off meeting where we could get to know the partners in person. Thanks to videoconferencing, which I had previously had very little experience with, there is a degree of familiarity in spite of the distance. We now have the cameras on for many of our meetings, making it feel at least a little bit like we are sitting across from each other.

Mr. Slatter, your company Sensitec produces sensor solutions. Can you briefly tell us what that means?

Slatter: Sensitec’s sensors are used for applications such as monitoring the condition of machines. Nowadays, many machine manufacturers no longer sell their machines per se, but rather a machine’s availability, i.e., the time during which the machine is used. In this business model, also referred to as an “availability-oriented business model”, a high level of availability is crucial. This means having the machines operate constantly and avoiding downtime caused by unplanned repairs. To this end, the machines are monitored using sensors. Over the last four to five years ,we have noticed that our sensors are increasingly being used for condition monitoring of this sort. This was not a direct sales strategy on our part; we were simply swept along by the market and by demand.

What is it like to collaborate with Fraunhofer IIS as an application-oriented research institution, and what makes it so special? What expertise does your company bring to the table, and what is the contribution of Fraunhofer IIS?

Slatter: ZFor one thing, Fraunhofer IIS has valuable prior experience and achievements that we can build on. This means we don’t have to start from scratch. This past experience and knowledge are the result of the applied research that Fraunhofer IIS engages in. As a result, Fraunhofer IIS has a valuable overview of what the industry is looking for. For another, Fraunhofer IIS provides the physical hardware, the ASICs. These are equipped with artificial intelligence functions that are responsible for signal processing and evaluation. Another contribution by Fraunhofer IIS is the part it plays in the research project. Industry partners often have very concrete expectations of a project. Universities are primarily responsible for the basic research. They do also focus on application to a certain degree, but this has its limits. This is where Fraunhofer IIS comes into play, acting as a bridge between universities and industry. It fills the gap between the concrete expectations of industry partners and the theoretical knowledge of universities. This function is crucial and indispensable for research projects like this one. That’s why we have been collaborating with Fraunhofer IIS ever since our company was formed. We have also worked on several other BMBF and the AIF (German Federation of Industrial Research Associations) projects with various Fraunhofer institutes in the past, including Fraunhofer IISB and Fraunhofer IZM. Many of the research fields pursued by the Fraunhofer institutes coincide with our own goals and technologies, so we have enjoyed numerous successful collaborations with Fraunhofer. I am also convinced that Sensitec is a good project partner. We are still a relatively small company, so it’s important for us that the project has a meaningful outcome. Unlike larger firms, we have the benefit of short communication routes and direct contact.

What added value do you hope collaborating with Fraunhofer IIS will bring for your company?

Slatter: Although the project is still at a very early stage, implementing the results in end products is of the utmost importance for us. As soon as the project is completed, we will get started with product development. The more fruitful the preparatory work during the project, the sooner it can be transferred to industrial applications. Fraunhofer IIS is a huge support in this regard thanks to its proximity to industry. Its staff know which specifications must be met, whether adjustments are necessary for industrial applications, or whether further development is needed. This application-oriented approach to research leads to faster development. Accordingly, we expect that the product development phase will potentially only take two or three years rather than five. In previous projects, Fraunhofer IIS has helped us to accelerate and shorten the development phase. Another very positive effect of our collaboration on this project is the attention and interest that we receive as a partner in a BMBF project. For example, we have had industry bodies and associations asking us what exactly we are doing and whether the project has already yielded results. This level of interest is unusual. Many projects go by unnoticed, without eliciting external enquiries or interest. This attention is a very positive signal, and it confirms once again that this is a great project to be involved in.

Dr. Matthias Völker has been developing integrated circuits for sensor applications at Fraunhofer IIS since 2005. In 2016, he became manager of the Integrated Sensor Signal Processing (ISSP) group at the Smart Sensing and Electronics Division. His core interests lie in the design of low-power sensor ASICs with integrated signal processing, and the deployment of artificial intelligence methods directly at the sensor. In the BMBF-funded research project KI-PREDICT, this is achieved by integration of both the analog interface and digital evaluation capabilities with AI functions.

Dr. Markus Stahl-Offergeld has been working in the Magnetic Sensor Systems group in the Fraunhofer IIS Smart Sensing and Electronics Division since 2005. He is an expert in the development of sensor ASICs with multidimensional Hall sensors. He makes use of this technical expertise to supervise complex development projects for industrial customers and research projects devoted to refining Fraunhofer IIS technologies.

Rolf Slatter has been Managing Director of Sensitec GmbH since 2009. Sensitec is a global market leader for high-quality and innovative magnetic sensor solutions. The cooperation with Fraunhofer IIS accelerates the development of new and innovative solutions for Sensitec.