Intel has launched a $99 “Neural Compute Stick 2” AI accelerator built around a new Myriad X VPU that adds a Neural Compute Engine and more cores for up to 8x greater performance.

Intel may be scaling back a bit on its IoT business, but it continues to push hard with the Myriad neural network acceleration technology it acquired when it bought Movidius. Intel has just released its third-gen “Myriad X” technology for AI acceleration on edge devices, debuting on a $99 Intel Neural Compute Stick 2 (NCS2).

Intel Neural Compute Stick 2
(click image to enlarge)

Supported with an Ubuntu-based desktop SDK and Intel’s OpenVINO AI toolkit, the USB stick “affordably speeds the development of deep neural networks inference,” says Intel. The NCS2 “enables deep neural network testing, tuning and prototyping, so developers can go from prototyping into production leveraging a range of Intel vision accelerator form factors in real-world applications,” says Intel. Potential applications include smart cameras, 360 cameras, drones, industrial robots, security systems, VR/AR headsets, and smart home devices. The 72.5 x 27 x 14mm device is equipped with a USB 3.0 port and supports 0 to 40°C temperatures.

Intel announced the NCS2 at the start of its second Intel AI Devcon event, held in Beijing on Nov. 14 and 15. Intel will also showcase several applications based on the earlier Myriad 2 technology and supply new details on two AI technologies due in 2019: the Nervana Neural Network Processor (NNP) and the Intel DL Boost AI accelerator, which is built into its Cascade Lake generation of Xeon chips.

Shortly before Intel acquired Movidius in 2016, the company launched its first USB stick implementation of the Myriad 2 VPU technology, called the Fathom. This miniaturized version of the earlier Myriad 2 reference board is essentially the same technology that re-emerged as Intel’s Neural Compute Stick.

The NCS is now used by “tens of thousands” of developers and has spawned AI stick imitators such as Google’s Edge TPU Accelerator. The Myriad 2 technology also drives Aaeon’s new mini-PCIe form-factor UP AI Core acceleration module, which is available as an add-on to a new UP Core Plus SBC.

Myriad X (left) and UP AI Core mini-PCIe implementation of earlier Myriad 2
(click images to enlarge)

Announced back in Aug. 2017, the Myriad X VPU debuts a neural compute engine described as a “dedicated hardware neural network inference accelerator.” The neural engine helps provide the Myriad X with up to 10X higher performance than the Myriad 2 “for applications requiring multiple neural networks running simultaneously,” says Intel. An Intel slide deck claims the Intel NCS2 stick offers up to 8x higher performance “on deep neural networks.”

The VPU itself provides theoretical performance of 4 TOP (trillion operations per second). However, current real-world performance is 1 TOP, the same as found on the BM1880 TPU embedded in Bitmain’s Sophon BM1880 EDB (“Sophon Edge”) 96Boards SBC.

The Myriad X chip’s allotment of “SHAVE” cores have expanded to 16x cores. There’s also a high-throughput intelligent memory fabric that supports up to 400 GB/sec of internal bandwidth. The chip is touted for offering low power consumption, but we saw no details on that score.

Upgrades to the imaging and vision engines include expanded vision accelerators. For example, the new Stereo Depth accelerator can simultaneously process 6x camera inputs (3x stereo pairs) each running at 720p60 resolution. There’s also a new native 4K ISP pipeline with support for up to 8x directly connected HD sensors via 16x MIPI lanes. An optional version of the chip includes 4 Gbit DDR4.

 
An Ubuntu-powered dev kit and OpenVINO optimization

The Myriad X chip and Intel NCS2 can be programmed using an upgraded version of the Movidius Myriad Development Kit (MDK). Ubuntu 16.04.3 LTS is currently the only supported desktop platform, but there will soon be support for the CentOS Linux distro and Windows 10.

The Eclipse-based MDK provides vision, imaging, and deep neural network libraries, as well as reference vision processing pipeline examples, all supported with source code. The kit offers a tool-kit for porting popular deep neural network platforms such as Caffe and TensorFlow.

Intel NCS2 development workflow
(click image to enlarge)

In addition to the MDK, you get access to Intel’s OpenVINO computer vision toolkit, which is optimized for the Myriad 2 and X chips. OpenVINO is used to optimize pre-trained models prior to deployment (see diagram above). For final products, Intel recommends its new line of Intel Vision Accelerator Design Products, a family of edge accelerator cards that also includes boards based on Intel Arria 10 FPGAs. All these solutions are part of an Intel AI: In Production ecosystem.

 
Further information

The Intel Neural Compute Stick 2 with the new Myriad X VPU is available now starting at $99 without the memory option. More information may be found in Intel’s NCS2 and Myriad X product pages.