Thanks to a mathematical breakthrough, AI applications like speech recognition, gesture recognition and ECG classification can become a hundred to a thousand times more energy efficient. This means it will be possible to put much more elaborate AI in chips, enabling applications to run on a smartphone or smartwatch where before this was done in the cloud.
Running the AI on local devices makes the applications more robust and privacy-friendly: robust, because a network connection with the cloud is no longer necessary. And more privacy friendly because data can be stored and processed locally.
The mathematical breakthrough has been achieved by researchers of Centrum Wiskunde & Informatica (CWI), the Dutch national research center for mathematics and computer science together with the IMEC/Holst Research Centre from Eindhoven. The results have been published in a paper (by Bojian Yin, Federico Corradi, and Sander M. Bohté) of the International Conference on Neuromorphic Systems. The underlying mathematical algorithms have been made available open source.
Under supervision of CWI researcher and UvA professor cognitive neurobiology Sander Bohté, researchers developed a learning algorithm for so-called spiking neural networks. Such networks have been around for some time, but are very difficult to handle from a mathematical perspective, making it hard to put them into practice so far. The new algorithm is groundbreaking in two ways: the neurons in the network are required to communicate a lot less frequently, and each individual neuron has to execute fewer calculations.
“The combination of these two breakthroughs make AI algorithms a thousand times more energy efficient in comparison with standard neural networks, and a factor hundred more energy efficient than current state-of-the-art neural networks”, says principal investigator Sander Bohté.
Inspired by the human brain
Bohté’s inspiration and motivation comes from the incredibly energy efficient way that the human brain processes information (20 Watt). Computers that mimic the brain's neuronal networks have produced wonderful applications in recent years - ranging from image recognition, speech recognition, automatic translation, to medical diagnoses - but require up to a million times more energy than the human brain.
The spiking neural networks developed by Bohté and his research team differ from those already integrated in AI applications. “The communication between neurons in classical neural networks is continuous and easy to handle from a mathematical perspective. Spiking neurons look more like the human brain and communicate only sparingly and with short pulses. This however means that the signals are discontinuous and much more difficult to handle mathematically.”
New type of computer chip
To run spiking neural networks efficiently in the real-world, a new type of chips are needed. Bohté says that prototypes are already being developed. “All kinds of companies are working hard to make this happen, like our project partner IMEC/Holst Centre.”
Bohté's methods can train spiking neural networks comprised of up to a few thousand neurons, less than typical classical neural networks, but sufficient for many applications like speech recognition, ECG classification and the recognition of gestures. The next challenge will therefore be to scale up these networks to 100.000 or a million neurons, which will expand the application possibilities even further.
Listen to the podcast 'Superscience' in which Sander tells more about energy efficient AI (in Dutch)
More information
- Full paper: Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks, by Bojian Yin, Federico Corradi, Sander M. Bohté
- International Conference on Neuromorphic Systems
- Homepage Sander Bohté
- Homepage Bojian Yin
- YouTube video Machine Learning group CWI
Header image: Shutterstock.
