Sony is developing self-driving car technologies with ZMP, which sells autonomous RoboCar development platforms with Linux-based control and sensor systems.

Sony has turned to fellow Japanese company ZMP to develop a self-driving car, says the Financial Times (FT). Sony also invested 100 million yen ($842,000) in ZMP for a 2 percent share. The partners are not necessarily building a commercial self-driving car — FT says they will develop “self-driving car technologies.” But whether it’s a full car or an autonomous automotive imaging system, it will likely run on Linux. Since 2009, ZMP has been selling autonomous plug-in hybrid RoboCar development platforms that integrate Linux control and sensing systems.

RoboCar PHV
(click image to enlarge)

The original 17 x 7.7 x 8.4-inch “RoboCar 1/10,” which is still available and continues to run Linux on an AMD Geode processor, can’t include human passengers, but Barbie and Ken should fit in just fine. Since then, ZMP has introduced a golf cart sized RoboCar MV2, which is just big enough to carry a passenger, and it recently introduced two full-sized PHV and HV models based on a Toyota Prius. The firmware continues to be based on Linux (see farther below).


Earlier RoboCar MV2 (left) and original toy-sized RoboCar 1/10
(click images to enlarge)

According to FT, Sony is primarily interested in developing vision systems, or at the very least developing sensors that are optimized for other vision systems. FT quotes Shigeo Ohba, general manager of Sony image sensor group has saying: “Sony’s sensing technology has a potential to offer critical support to automotive manufacturers.”

The automotive industry is hesitant to see big tech companies enter its turf, and with Sony’s decline over the past decade, the consumer electronics firm is not likely in a position to follow through with a full autonomous car. Google, which has more funds for a major automotive effort, is struggling to find partners for its self-driving car. Apple, another well-heeled tech giant, is “studying” self-driving car technology, according to a Feb. 14 Reuters report .

Sony owns 40 percent of the CMOS sensor market, followed by OmniVision and Samsung, each with about 16 percent, says FT, citing a study by Techno Systems Research. Although Sony sensors are in many digital cameras and smartphones including the iPhone, it only has 5 percent of the automotive sensor market, behind market leaders Aptina and OmniVision.

Luxury cars, and even mid-range models, are increasingly integrating ADAS (Advanced Driver Assistance Systems) technologies, which require more sensors for vision, telematics, and more. Semi-autonomous and self-driving cars will boost that number significantly, especially with vision systems. According to FT, self-driving cars could include as many as 10 cameras.

There’s a growing consensus across the robotics world from drones to home and industrial robots to autonomous cars that the future for most autonomous vehicles is not in expensive technologies like LIDAR (Light Detection and Ranging), which has been used in Google’s Ubuntu-driven self-driving car, but in lower-cost vision systems. Thanks to faster processors and more sophisticated algorithms, vision systems, which are also used on the Google car, are capable of increasingly advanced obstacle avoidance.

Many believe the cameras will need to be augmented with GPS-assisted wireless communications with other cars (vehicle-to-vehicle, or V2V) and/or vehicle-to-infrastructure (V2I), also known as “drive-by-wire.” ZMP supports drive-by-wire, which is less common in the U.S., as well as LIDAR and other technologies, but its main focus appears to be its Linux-based RoboVision sensing system.

The problem with today’s imaging sensors is they are often challenged in identifying objects in harsh lighting conditions, or in the rain and snow. Night vision also remains a challenge, although much more progress has been made here.

In October Sony announced a series of imaging sensors for automotive cameras due by year’s end that it claims can capture quality color images on a moonless night, says FT. The company has also boosted internal investment in automotive sensors prior to the ZMP investment.

 
ZMP’s RoboCar PHV/HV

Sony and ZMP will likely use ZMP’s most recent RoboCar PHV and very similar RoboCar HV cars to test new imaging systems and other self-driving technologies. Like the original Google car, the test cars are built on a Toyota Prius. (The previous golf-cart sized RoboCar MV2 announced in 2012 was also based on a Toyota vehicle, in this case a 2.0 x 1.6 x 1.0-meter electric golf cart.)

RoboCar PHV evolution
(click image to enlarge)

ZMP does not say which processor the RoboCar PHV/HV cars use to run Linux, but it appears to be an Intel Core i7, which is used on an IZAC [translated] (Intel ZMP Autonomous Controller) system announced last summer, which looks like it’s used by the RoboCar.

The translated RoboCar PHV/HV product page, has some gaps, but it appears that the only main difference between the 14 million yen PHV and the 12 million yen HV is the PHV’s addition of a GUI touchscreen that’s compatible with tablets and smartphones. There’s also a 7 million yen PHV Primitive model that lacks human driving capabilities, but is the only model rated to drive on certain public drive-by-wire test roads equipped with roadside sensors.

RoboCar PHV detail
(click image to enlarge)

The RoboCar PHV/HV’s Linux SDK includes “libraries and documents,” says ZMP. The CAN interface and protocol, which controls most of the cars communications, is said to be “open,” and is controllable by user systems such as DSPACE.

In addition to offering manual driving functions (aside from the Primitive model), the RoboCar PHV/HV provides autonomous speed, brake, accelerator, shifting, and pedal controls, complete with an override switch. The cars can self-steer according to a user-specified torque and angle.

RoboCar PHV hardware (left) and software architecture
(click images to enlarge)

Besides offering the Prius’s standard safety features, the RoboCar PHV/HV cars supply safety features like an emergency stop switch, the ability to check CAN error parameters with limited values settings, and self-detection of hardware errors.

Options include a wide variety of localization and sensing devices on which vendors can test out their equipment. These include an intelligent GPS system, as well as basic acceleration, gyro, geomagnetism, atmospheric pressure, temperature, and humidity sensors. Bluetooth transceivers are also available.

RoboCar PHV GUI
(click image to enlarge)

More advanced sensors on the RoboCar PHV/HV include the company’s Linux-based RoboVision for Car2 stereo camera system, which will likely be a focus of the Sony research. There’s are also optional 2D and 3D LIDAR systems, a millimeter-wave radar system, a tilt inclinometer, and more.

The cars are also touted for their Microsoft Azure cloud interface, as well as ZMP’s advanced path generation, which offers fine-path control. The latter is said to enable more human-like performance when following curves, turning corners, or changing lanes. (A relatively complete spec list is available in this translated PDF datasheet, which also identifies the system’s Linux underpinnings.)

ZMP also offers some educational robots, including a humanoid Nuvo Humanoid. However, this appears to run on a Windows based stack.

 
Further information

The RoboCar 1/10 and MV2 appear to be available now, while the RoboCar PHV/HV appear to be set for release this summer. More information may be found at the ZMP website.