Silk Labs has won KickStarter funding for its “Sense” home surveillance camera and automation hub, which features voice, face, and gesture recognition, and AI.

Silk Labs has found Kickstarter success for its first home automation product. The Linux-based Sense smart camera is available through Mar. 17 starting at $249, and will ship in December. The startup was formed last year by former Mozilla CTO Andreas Gal, along with Chris Jones, who co-developed Firefox OS with Gal, and Michael Vines, a former senior director of technology at Qualcomm.

Three views of the Silk Labs Sense
(click image to enlarge)

The silky sleek-looking Sense device is a home surveillance camera with a self-learning AI engine that can “detect your presence, recognize your voice and gestures, learn your preferences, and choreograph your connected devices to respond to you appropriately,” according to Silk Labs. In addition to providing basic home surveillance services such as giving you a remote video feed of your pet, it studies a user’s preferences and routines to automatically adjust lights and temperature, and launch music based on time and context.

To control Sense, you can use a smartphone app, as well as direct voice and gesture commands. As with most such products, Sense is touted for its quick and easy setup.

Sense details
(click image to enlarge)

Sense’s inputs are limited to a microphone and a wide-angle, 130 degree FoV 1080p camera with an infrared night vision sensor. However, Sense can act as a smart hub for third-party home automation devices, or as a companion to an existing hub. The smart camera, which integrates dual-band WiFi 802.11a/b/g/n and Bluetooth 4.0 with BLE, will work with “lighting solutions such as Philips Hue and LIFX, sound systems including Sonos, home systems like the Nest thermostat, and many more to come,” says Silk Labs. The website also lists Amazon’s Echo.


Sense’s camera has a 130 degree FoV
(click image to enlarge)

Last year, Silk Labs CEO Gal told CNET the platform would work with Alphabet-backed Nest’s Weave home automation connectivity platform, as well as the AllSeen Alliance‘s AllJoyn IoT ecosystem. There’s no mention of these technologies, however, and there’s no integral support for the Weave-associated, 6LoWPAN-based Thread mesh wireless technology, or even ZigBee or Z-Wave.

The device runs on an unnamed, 1.4GHz hexa-core ARM SoC with a “high performance, ~100 GFLOPS GPU for compute-heavy machine learning” plus 2GB of RAM and 16GB of flash. This is according to Gal, CEO of Silk Labs, and Philipp von Weitershausen, Silk Labs software engineer, in an email reply to LinuxGizmos. A separate announcement cites a 500+ GFLOPS GPU, and also notes 1GB RAM with 8GB flash, but Gal and von Weitershausen confirmed the 2GB RAM and 16GB flash specs listed on the KS page. The curvy, 5.16 x 2.13 x 1.85-inch device is further equipped with a speaker, reset button, accelerometer, ambient light sensor, and a micro-USB port for power.

Sense uses the startup’s new Linux-based Sense OS distribution. “At the heart of the Silk platform is Linux kernel, device drivers for WiFi, Bluetooth, etc., and a few user space daemons,” said Gal and von Weitershausen. “In essence these are the same things you would find on an Android smartphone minus the Java framework and applications. It is similar to the base system that Google’s Brillo appears to be using, as well as the base system used by Firefox OS. On top of this Linux base system we run the Silk framework, which is a combination of native code as well as JavaScript running in Node.js.”

There’s no indication of open source licensing. However, Kickstarter users will have access to a free, Node.js based SDK. “We will have a way to make third-party extensions discoverable to users, and we want to empower developers to distribute their extensions,” explained Gal and von Weitershausen. “It will feel a bit different than an app store but fulfill a similar role.”

Sense processes sensor inputs locally rather than depending on cloud service, enabling operation even if there’s an outage on the home WiFi or cloud datacenter. Although processing is local, there is a cloud service “to notify users of things going on in a house via notifications and to store and stream encrypted video from events detected at home, etc.,” explained Gal and von Weitershausen, “Some of those services such as long-term storage may be provided for a subscription fee. There will be OS updates after launch, for security fixes and new features, and they will always be free.”

The connection between Sense and an authorized smartphone is encrypted using a secure Bluetooth LE channel. “By touching Sense with their phone and passing a proximity gate on either side, the user pairs the two,” said Gal and von Weitershausen. “Sense and the phone exchange encryption keys so they can continue to communicate using end-to-end encryption from that point forward. That allows the phone to securely transmit configuration data such as the WiFi password, and Sense to transmit live video and other recorded data back.”

Sense can “tell people apart and can detect who is home at all times,” according to Silk Labs. There’s no indication of how it handles multiple users with conflicting requirements, but this seems to be an inherent challenge for all such anticipatory automation gear.

Left to right: Admobilize Matrix, Nest Cam, and D-Link DIY Security Kit cam
(click images to enlarge)

Other somewhat similar devices include AdMobilize’s retail-oriented Matrix hub, which is touted for its computer vision capabilities. Sense will compete more directly with Linux-based smart home surveillance cameras like Nest’s Nest Cam and D-Link’s DIY Security Kits.

We still feel a bit queasy at the idea of a continually surveilled home environment, even if it is secure from hackers. Yet, home cameras are increasingly popular, especially among apartment dwellers with pets and kids.

What’s intriguing about Sense is that it promises to reduce the number of notifications and superfluous information bombarding your smartphone while instead acting intelligently behind the scenes. We’re skeptical that we’ve reached the point where a computer can truly understand what one person wants, let alone a family, but if it’s done properly without overreach, this seems to be the right direction for the home automation category.

Sense overview video

 
Further information

The Silk Labs Sense device is available on Kickstarter through Mar. 17 starting at $249, with shipments due in Dec. 2016. More information may be found at the Sense Kickstarter page as well as the Silk Labs developer page.