In a technical guide published by Hugging Face and NXP, engineers demonstrated a three-stage pipeline to train and optimize Vision-Language-Action (VLA) models on low-power hardware. By deploying these policies to the NXP i.MX 95 applications processor, the team established a blueprint for bringing real-time robotic controls directly to embedded chips. Engineers successfully reduced ACT model inference latency on the NXP i.MX 95 SoC from 2.86 seconds to 0.32 seconds.
Architectural Decomposition and Quantization
Running complex multimodal models on embedded robotics requires moving away from monolithic network execution. The NXP and Hugging Face team partitioned the SmolVLA model into three decoupled logical stages: vision processing, LLM backbone reasoning, and action expert denoising.
Each subsystem was optimized independently based on its sensitivity to precision loss:
- Vision Encoders & LLM Prefill: Quantized down to 8-bit and 4-bit configurations with minimal impact on decision accuracy.
- Action Expert Denoising: Preserved at higher precision, as iterative flow-matching steps tend to accumulate quantization errors rapidly across execution cycles.
- Hardware Acceleration: Workloads were routed across the i.MX 95 SoC's integrated eIQ Neutron NPU and ARM Cortex CPUs.
Dataset Practices and Asynchronous Control
Deploying VLA models effectively requires solving physical control latency. In standard synchronous loops, the robotic arm halts while waiting for model inference, producing jittery, oscillatory motions.
To establish fluid motion, NXP implemented asynchronous control loops where action chunk generation runs concurrently with physical execution. This strategy requires end-to-end inference time to remain strictly shorter than the action chunk execution duration. Data collection best practices included using three cameras—top, left, and a dedicated gripper-mounted camera—along with workspace spatial clustering and 20% recovery trajectory episodes to handle unexpected errors.
Performance Benchmarks and Real-World Trade-offs
Testing on a standard manipulation task ("Put the tea bag in the mug") demonstrated significant speedups across tested architectures on the i.MX 95 processor. While full FP32 precision models yielded high accuracy, optimized variants drastically improved control throughput.
Key performance results from the evaluation include:
- ACT Baseline (FP32): Achieved a 2.86-second latency with 96% overall task accuracy across 30 evaluation episodes.
- ACT Optimized: Reduced latency to 0.32 seconds while preserving an 89% overall success rate.