Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module uses an M.2 2280 interface and is designed to handle local AI inference, including large language models, vision-language models, and computer vision workloads on embedded Linux and Android systems.

The RK1820 and RK1828 are dedicated AI coprocessors rather than standalone application processors. Both provide an NPU rated at 20 TOPS (INT8), with support for INT4, INT8, INT16, FP8, FP16, and BF16 numerical formats. The RK1820 integrates 2.5GB of DRAM, while the RK1828 increases this capacity to 5GB.

The memory uses a 3D stacked architecture that places DRAM in the same package as the processing hardware. This arrangement is intended to address the bandwidth requirements of language-model inference while reducing dependence on the host system’s external memory.

RK1820/RK1828 M.2 Computing Card

The accelerator uses a standard M.2 2280 M-Key form factor with a single PCIe 2.1 lane for host communication. Supported Rockchip host processors include the RK3568, RK3572, RK3576, and RK3588. The coprocessor architecture allows AI inference to run on the accelerator while the host processor handles application logic, peripheral interfaces, and other system tasks.

Supported AI workloads include convolutional neural networks, object detection, audio processing, time-series prediction, large language models, and multimodal models. The accelerator is designed for offline inference with models in the 3B to 7B parameter range, including examples from the Qwen and Llama families.

More specific performance figures are available in a Forlinx blog post. With an RK1828 connected to the OK3588-C development board over PCIe, the reported results reach approximately 102 tokens per second for Qwen2.5-3B and 70 tokens per second for Qwen2.5-7B. The measurements use 128 input tokens and 128 generated tokens in performance mode, so actual throughput will depend on the model and operating conditions.

Forlinx has also demonstrated a multi-accelerator configuration using its OK3588-C development board with four RK1828 cards connected through PCIe. The setup uses pipeline parallelism to distribute Transformer layers across the accelerators, allowing larger models to execute using the combined memory resources.

The four-card configuration supports 27B and 31B parameter models, with a reported total accelerator power consumption of approximately 40W. In one manufacturer-reported 27B model test involving Python code generation, the system achieved a first-token latency of 819ms and generation speeds of up to 13 tokens per second.

A separate retrieval-augmented generation test using technical documentation produced approximately 13 tokens per second with an initial response time of around 1.2 seconds. These figures are based on Forlinx’s testing rather than independently verified benchmarks.

OK3588-C + 4x RK1828 PCIe Cascade

The accelerator uses Rockchip’s RKNN3 SDK for model conversion, quantization, and deployment. The development environment covers CNN, LLM, and VLM workloads, with documentation describing model deployment using the OK3588-C development board.

The software ecosystem supports models originating from TensorFlow, PyTorch, and ONNX, while Linux and Android are listed as supported host operating systems.

Further Information