Useful Sensors indicates that this device is built around the RockChip 3588S SoC with 8GB DRAM memory and NPU already seen in other SBCs such as the Indiedroid Nova, NanoPi R6C, etc.

They also mention that the AI in a Box is developed using open-source models like Whisper and Llama2. Moreover, all the code is made available under an open-source license, including their library for optimized transformer inference on the RockChip NPU.

RK3588S block diagram
(click image to enlarge)

These are some of the capabilities of the AI in a Box:

• Live Captions: Real-time closed captions based on audio input, viewable on the device’s screen or through HDMI for external displays.
• Conversational AI: Text-based interactions with a local Large Language Model (LLM) without the need for setup or API fees, covering various conversational topics.
• Live Translation: On-the-fly translation between major languages, with split-screen captions displaying both the original and translated text.
• Air-Gapped Privacy: Software runs solely on the device, ensuring data privacy. No Wi-Fi or Bluetooth connectivity, access via physical keyboard or Ethernet cable only.
• Voice Keyboard: Included USB Type-A cable converts nearby conversations into keystrokes for integration with other systems.
• Open Source: Built on open source models, offering code transparency for security and privacy audits, and supporting the development of real-time voice input applications in Python.

The video above demonstrates the real-time translation capability of the AI in a Box. Additional technical information may be found on the company’s GitHub.  

The company recently provided an update related to their prototype daughter board which comes with a 5” LCD screen, mic, speaker and USB emulation. 

AI in a Box Prototype daughter board
(click image to enlarge)

Useful Sensors states that “ the finished product may not produce results that are accurate or fast enough for all potential use cases.” and that “All current AI language models are very good at sounding confident while giving totally wrong answers. This means that you can’t entirely trust any output from the device when you ask it factual questions, so it shouldn’t be your only source of information. “

AI in a Box (left) and Prototype Kit (right)
(click images to enlarge)

Further information

The AI in a Box Prototype Kit is offered at $475.00, which includes the Rockchip-based device along with a 3D printed enclosure, an external display, microphone, speaker, HDMI cable, USB cable, and a power adapter. Additional charges of $8.00 apply for shipping within the US and $18.00 for worldwide shipping. 

The AI in a Box device alone is available for $299.00 with similar shipping fees.