Autonomous driving middleware deployed across tens of thousands of vehicles

Problem

What the vehicle platform needed from its middleware: real-time perception and driver-monitoring workloads spread across multiple ECUs and compute engines, strict latency budgets, automotive safety requirements, and many teams building on the same foundation.

Approach

What was developed was a scalable middleware framework that could perform across multiple vendor ECU’s and provide a singular API for developers to use.

• Middleware for heterogeneous compute. The middleware was designed for multi-node coordination, IPC, and kernel execution across CPU, GPU, and specialized accelerators (NVIDIA VIC and PVA). The framework was modular across NVIDIA and custom silicon.
• GPU-accelerated inference. GPU preprocessing kernels were developed for real-time inference: image resizing, cropping, normalization, and tensor conversion. Inference runtimes and embedded vision pipelines were built with NVIDIA DRIVE OS, CUDA, and TensorRT. The work also included leading solution design for GPU-accelerated workloads, including the production deployment of LSTM models.
• Cross-functional delivery. The work involved collaboration with the Perception, Platform, Camera, and Driver Monitoring teams.
• Diagnostics and performance. Internal tooling was built for real-time streaming diagnostics, covering buffer lifecycle tracking, latency analysis, and GPU compute-time profiling. Distributed applications were debugged and optimized across multiple ECUs, resolving millisecond-level latency issues.
• Testing and safety. Unit, integration, and hardware-in-the-loop (HIL) tests were written, and static analysis was applied to meet automotive safety requirements.
• Simulation and visualization tooling. Visualization tools were built for inspecting and validating input sensor data and AI inference outputs.

Result

The middleware runs in production across tens of thousands of vehicles today.