About

Building perceptive robots means solving a critical bottleneck: processing massive streams of sensor data while running demanding AI models on a compact platform.

Neocortex is a dedicated Sensor Processing Unit (SPU) engineered to serve as the sensory backbone for robots. The key to Neocortex’s design is the separation of compute. Rather than relying on a single processor, we divide the workload across an IO-aligned processor and a GPU-aligned processor.

A power-efficient LattePanda Mu board manages the high-bandwidth sensor processing and core logic, while a dedicated Jetson device handles the heavy lifting for GPU-intensive tasks. This enables us to provide realtime first-person and third-person views of a robot without worrying about stalling key user processes. Paired with a microphone array, operators or AI models will have both audio and visual spatial information of their surroundings.

Architecture

Compute Nodes

• LattePanda Mu - Primary processing unit handling 4 camera feeds plus audio I/O
• Nvidia Jetson Orin NX - Secondary GPU-accelerated processor

Camera Array

Each of the 4 cameras connects through dedicated USB 3.0 controllers to the LattePanda Mu. By using separate USB controllers for each camera rather than a single USB hub, we can avoid typical USB bandwidth limitations. This allows us to use high resolution and high frame rate USB cameras without degradation, keeping software and BoM complexity low.

Audio System

The microphone array is connected via a USB 2.0 controller to prevent mixing of USB 2.0 devices with the USB 3.0 controllers utilised in the camera array. The speaker output via TRS line (audio jack) on the microphone array allows for hardware echo-cancellation, which can be further assisted by software implementations.

Deploying applications

Just like deploying an application in the cloud via a Dockerfile, GPU-constrained tasks can be containerised and deployed to the onboard Jetson module.