Google unveiled a stripped-down “Edge TPU” version of its Tensor Processing Unit AI chip plus a Cloud IoT Edge stack for Arm-based IoT gateways that run Linux or Android Things. An NXP-based dev kit is due in October.

Google announced a lightweight, embedded version of its enterprise focused Cloud Tensor Processing Unit (Cloud TPU) AI co-processor. The new Edge TPU ASIC is similarly optimized for Google’s TensorFlow machine learning (ML) framework. It’s designed to run TensorFlow Lite ML models on Arm Linux- or Android Things based IoT gateways connected to Google Cloud services that are optimized with Cloud TPU chips. A development kit due in October will use an NXP SoC.

Edge TPU chip (left) and Edge TPU development kit
(click images to enlarge)

Google also announced a Cloud IoT Edge stack that integrates the Edge TPU assisted gateways with Google Cloud to enable cloud-integrated IoT edge computing and analytics. The combined platform “lets you build and train ML models in the cloud, then run those models on the Cloud IoT Edge device through the power of the Edge TPU hardware accelerator,” says Google. “You can accelerate ML training in the cloud, then have lightning-fast ML inference at the edge” to enable “local, real-time, intelligent decisions.”


Cloud IoT Edge workflow
(click image to enlarge)

The Edge TPU and Cloud IoT Edge platform competes with Amazon’s AWS IoT framework and related, Linux-based AWS Greengrass stack for bringing cloud analytics to the edge. As with Greengrass and Microsoft’s upcoming Azure Sphere platform for IoT, the architecture is designed to enable faster, local decision making by avoiding the latency and security risks of cloud communications.

Unlike Greengrass, however, there are no claims that you can run the Edge TPU-assisted edge analytics independently without a sustained cloud connection. Like Azure Sphere, Google’s platform also includes a defined hardware component. However Azure Sphere does not appear to include an AI co-processor on the edge.

The tiny Edge TPU chip is a lightweight version of Google’s server-oriented Cloud TPU chips. The ASIC enables concurrent execution of multiple AI models per frame on a high-resolution video, at 30fps, says Google. Equipped with PCIe and USB interfaces, as well as Int8 and Int16 numerics, the Edge TPU is said to be optimized for power efficiency,

Cloud TPU modules
(click image to enlarge)

The Cloud IoT Edge framework supports GPU- and CPU-based accelerators in addition to working with the Edge TPU chip. The software “extends Google Cloud’s powerful data processing and ML capabilities to gateways, cameras, and end devices, making IoT applications smarter, more secure and more reliable,” says Google.

Cloud IoT Edge includes a runtime for Linux or Android Things gateways, as well as an “Edge IoT Core runtime that more securely connects edge devices to the cloud, enabling software and firmware updates and managing the exchange of data with Cloud IoT Core,” says Google. There’s also a TensorFlow Lite based Edge ML runtime “that performs local ML inference using pre-trained models, significantly reducing latency and increasing the versatility of edge devices.”

 
Edge TPU development kit

In October, Google will launch a sandwich-style development kit to jumpstart Cloud IoT Edge and Edge TPU development. The kit is built around an unnamed computer-on-module with an Arm-based NXP SoC and an Edge TPU chip. There’s also a roughly Raspberry Pi sized baseboard that appears to include Ethernet, HDMI, USB, and 3x micro-USB ports. It also appears to have an RPi-style 40-pin GPIO connector.

Edge TPU development kit (left) and differing task assignments for Cloud IoT Edge/Edge TPU vs. Google Cloud
(click images to enlarge)

Presumably, the kit will run Android Things, although it may also support Linux. The module in the image looks nothing like the four production modules for Android Things 1.0 from Innocomm, Intrinsyc, and MediaTek. We also searched our stories about NXP-based Arm modules, but also came up empty. Maybe next time we’ll use TensorFlow to find the image for us.

Listed partners include NXP, Arm, Adlink, and Nexcom, among others. Arm, which has also partnered with Microsoft on Azure Sphere, has its own competing MLP co-processor as part of its Project Trillium for AI and machine learning.

 
Further information

An early version of the Cloud IoT Edge software, Edge TPU chip, and Edge TPU development kit will be available in October. You can apply for early access to the Cloud IoT Edge Alpha and Edge TPU Early Access development boards from Google’s Edge TPU and Cloud IoT Edge product pages. More information may be found in Google’s announcement.