Qualcomm announced a deep learning toolkit for implementing neural processing and other AI functions directly on devices that integrate Snapdragon 820 SoCs.
The “Snapdragon Neural Processing Engine” is Qualcomm’s first deep learning software development kit for devices based on its Snapdragon 820 SoCs. The SDK, which is due for release in the second half of 2016, brings the company’s “Zeroth Machine Intelligence Platform” to Snapdragon 820 based devices.
Qualcomm’s SDK was one of two solutions introduced in conjunction with this week’s Embedded Vision Summit in Silicon Valley, that allow running advanced AI algorithms directly on mobile or embedded devices without relying on cloud processing. The other is the Movidius Fathom USB stick, which is billed as “the world’s first embedded neural network accelerator.”
Both solutions should come in handy in medical, automotive, drone, robotics, and other applications in which fast, reliable response time and privacy are prioritized. Eventually, technologies such as these will enable the implementation of deep learning technologies directly on embedded, mobile, and IoT devices without requiring access to a cloud-based neural network.
Qualcomm Cognitive Computing graphic
Qualcomm’s SDK for Qualcomm’s Zeroth Machine Intelligence Platform is the “first deep learning toolkit optimized for mobile.” says the company. Specifically, it’s optimized for the Snapdragon 820, and presumably also the almost identical automotive version, the Snapdragon 820A, for which Zeroth tools have previously been promised.
This Snapdragon Neural Processing Engine SDK will enable OEMs to run custom neural network models on 820-based devices such as smartphones, security cameras, automobiles, and drones, all without a connection to the cloud, says Qualcomm. The SDK can enable features like scene detection, text recognition, object tracking and avoidance, gesturing, face recognition and natural language processing. Applications are said to include healthcare, automotive, security, and imaging.
The SDK provides an accelerated runtime for on-device execution of convolutional and recurrent neural networks, says Qualcomm. The software supports common deep learning model frameworks, including Caffe and CudaConvNet. Qualcomm had no details on OS support, but presumably Linux and/or Android will lead the way.
Snapdragon 820 functions
The SDK makes use of the Snapdragon 820’s 624MHz Adreno 530 GPU and Hexagon 680 DSP in addition to its CPU block. The latter is comprised of four 14nm FinFET fabricated “Kyro” cores that roughly mimic the ARMv8 Cortex-A72. The Kryo cores — two at 2.2GHz, and two at 1.6GHz — offer up to twice performance and twice the power efficiency of Cortex-A57 cores, says Qualcomm. The SoC also features a 14-bit Spectra image signal processor (ISP) for up to 25-megapixel cameras.
The underlying Snapdragon-optimized Zeroth Machine Intelligence Platform can be found in visual intelligence software such as Snapdragon Scene Detect, a feature that can analyze and classify recognizable image types and categories. Zeroth also drives malware detection algorithms in Snapdragon Smart Protect, which can detect zero-day and transformed malware invisible to typical anti-malware solutions. Using real-time machine learning-based behavioral analysis, Snapdragon Smart Protect can “detect new malicious apps before their signature files have been incorporated into the signature databases of traditional AV apps,” says Qualcomm.
One of the first customers for Qualcomm’s Zeroth-based SDK is autonomous vehicle technology firm NAUTO.
“The Qualcomm Snapdragon Neural Processing Engine SDK helps us bring deep learning to our connected camera and smart cloud network, and that helps us present a full, contextual picture of the driving environment to our commercial automotive fleet and auto insurance customers,” stated Frederick Soo, CTO of NAUTO. “The Neural Processing Engine SDK means we can quickly deploy our proprietary deep learning algorithms to our Snapdragon-based connected camera devices in the field, which can detect driver distraction and help prevent auto accidents.”
More details are available in Qualcomm’s Snapdragon Neural Processing Engine announcement.