Artificial intelligence and the Internet of Things seem to be a match made in heaven, yet there are many hurdles to overcome to perfectly marry these two technologies. In general, IoT is meant to be a low-power, battery-powered type of technology. On the other hand, artificial intelligence, specifically convolutional neural networks (which are essential for machine vision), is extremely computationally expensive.
Edge computing provides real-time data processing. Image used courtesy of Theory of Computation
To overcome these challenges, the general practice has been to offload computation to the cloud. This, however, comes with a whole host of latency and security issues. A self-driving car, for example, needs to make real-time decisions as quickly as possible. It simply can’t afford to wait for data to be sent to the cloud, computed on, and returned.
But what about embedded devices? Kris Ardis, the executive director of the Micros, Security, and Software Business Unit at Maxim Integrated, posits that currently, small embedded devices are “missing the rest of this AI revolution.”
“They can’t see and they can’t hear—much more than simple words,” Ardis explains. “And so that’s the gap that we’re trying to fill: how can we bring more of that AI promise to the embedded universe?”
A New Solution: Maxim Integrated’s NN Accelerator Chip
To answer this question, Maxim Integrated turned to low-power, high-performance chips for artificial intelligence. This morning, the company announced its newest product: a neural network accelerator chip meant to enable AI in battery-powered IoT devices.
The new chip, the MAX78000, consists of two ultra-low-power cores—the Arm Cortex-M4 core or a RISC-V core—an FPU-based microcontroller, and a convolutional neural network accelerator. Ardis comments, “That RISC-V is there because it’s a nice low-power way to massage data if necessary before it gets into the accelerator.”
From a performance perspective, Maxim claims some impressive specs—specifically in terms of power and latency.
A simplified block diagram of MAX78000. Image used courtesy of Maxim Integrated
With respect to energy, the company says that the MAX78000 provides:
- 1,100x lower energy consumption when running MNIST
- 400x latency improvement on MNIST
- 600x low energy consumption during keyword spotting compared to a low-power Cortex M4F
- 200x improvement on keyword spotting compared to a 96 MHz Cortex M4F
Let’s dive into how exactly these specs are achieved.
The Heart of MAX78000: The Neural Network Accelerator
The most unique feature in this SoC is the neural network accelerator, which is specialized hardware designed to minimize the energy consumption and latency of convolutional neural networks (CNNs).
According to Ardis, the architecture employed is entirely proprietary and novel. It was designed with the goal of minimizing data movement, which is well known to be a significant burden to on-chip energy, especially in processing complex math configurations in CNN chains.
In addition, the accelerator—which supports common tools in the machine learning universe like TensorFlow and Pytorch— is designed to increase mathematical parallelism, optimizing energy expenditure and significantly decreasing inference time.
Overview of important functional blocks. Image used courtesy of Maxim Integrated
Another feature of the system operation is that the microcontroller has minimal involvement. Generally speaking, the MCU in this architecture is meant to configure the network, load the data, and start it. After the MCU does its initial job, it essentially gets out of the way. This, too, proves to be extremely important for energy efficiency.
The devices also loads data before execution, which negates the need to access memory during inference, reduces energy consumption, and improves latency. “There’s no external memory required, which is actually one of the ways we save energy. All the memory is on-chip,” says Ardis.
A Game Changer for IoT?
This news looks to be extremely significant in the IoT space as the demand for low-power, high-performance AI chips mounts.
Depiction of the new NN accelerator. Image used courtesy of Maxim Integrated
According to Maxim Integrated, this device could potentially afford systems the ability to perform real-time decision making at the edge—faster than cloud-based computing—and without the associated security concerns.
The device could enable new applications, like facial ID within milliseconds or data-processing hearing aids, by bringing AI to edge devices. In this way, the MAX78000 may be a significant step in “cutting the power cord” for embedded devices.
Ardis expressed hopes that the MAX78000 may blaze a trail toward a type of “embedded revolution,” similar to the type of embedded revolution microcontrollers brought to the table. “Before the time of micros, nobody thought of all the things that the microcontroller will be. And now I’m wearing at least two,” he said.
“That’s what we think AI at the edge technology will turn into and hopefully, we’ll be one of the ones leading the way.”