“The whole team is delighted,” said the project leaders Dr. Marco Breiling from Fraunhofer IIS as well as Professor Dietmar Fey and Dr. Marc Reichenbach from Friedrich-Alexander-Universität Erlangen-Nürnberg. Breiling went on to explain that: “The prize money awarded for the follow-up project will provide further impetus for the development of neuromorphic hardware as part of our Next Generation Computing initiative. At our Erlangen and Dresden locations, we want to ensure that Germany plays a leading role in the practical applications of AI.”

“Artificial intelligence still consumes too much energy, but it has enormous potential to bolster Germany’s reputation as a hub for business and innovation and is increasingly finding its way into our everyday lives. With this in mind, the Federal Ministry of Education and Research is stepping in to provide funding and has called on higher education institutions and research establishments across the country to apply with their best ideas for developing an energy-saving AI chip. I’m delighted for the four winning teams selected today, who now have a unique opportunity to develop their projects further with funding of around a million euros each,” explained the Federal Minister of Education and Research.

Putting the signal processing to sleep

The particularly energy-efficient processing of time-series signals is addressed in the Lo3-ML project, whose name stands for “Low-power low-memory low-cost ECG signal analysis using ML algorithms.” This work involved developing a chip for AI calculations – or, more precisely, for deep learning inference – in order to analyze ECG signals with a view to determining whether a patient is healthy or suffering from atrial fibrillation. The key lies in putting part of the chip – specifically, the signal processing technology – “to sleep” while it isn’t needed, resulting in energy savings of up to 95 percent. The circuit can also be used for other applications that involve processing time-series signals, such as for anomaly detection or predictive maintenance.

Several innovative concepts in the chip

By using special systolic arrays, the researchers were able to significantly reduce the energy required for control logic. The technology uses only the three weights -1, 0, and +1, as well as batch normalizations with powers of two, paving the way not only for a highly versatile algorithm but also for the utmost efficiency when it comes to implementing the neuron’s calculations or normalization. Inside the chip is a form of non-volatile memory known as resistive RAM (RRAM), along with ultra-low-power read/write circuits that store precisely these ternary values with a high level of efficiency. The non-volatile memory allows the signal processing to be put to sleep for long periods of time so that it consumes no energy. After all, with signal bandwidths typically in the (sub-)kHz range, data arrive at the chip much more slowly than they can be processed. These data are therefore collected in a section of the chip that is awake while the AI algorithm sleeps and then AI-processed rapidly once the algorithm wakes up – at which point all parameters are available immediately thanks to the RRAMs. This approach can achieve energy savings of up to 95 percent relative to an “always-on” system.

In addition to digital and analog circuit development and ASIC implementation, the project placed great importance on automating the design process with a view to optimizing the AI algorithm and the circuit jointly in terms of their energy consumption. These efforts also called for newly developed tools for hardware-aware training of neural networks, such as for the extreme quantization applied here. This process can also be applied to other problems and applications.

The project offers an impressive demonstration of the potential for optimization that can emerge from close collaborations between algorithm designers and digital or analog chip developers.

“Energy-efficient AI systems” pilot innovation contest

A prize also went to the Fraunhofer Institute for Industrial Mathematics ITWM and TU Kaiserslautern for a project entitled “Holistic approach to optimizing FPGA architectures for deep neural networks via AutoML – automated machine learning” (HALF).

Applications to take part in the contest were received from 27 teams from higher education institutions and research establishments. Of those, 11 teams were selected for the final round. Each of the four winning projects receives an exclusive opportunity to submit a research project to implement their idea with application partners with funding to the tune of 1 million euros.