NVIDIA's KGMON LLM Agent Research Team achieved first place on the DABStep data analysis benchmark using a novel architecture built on the NeMo Agent Toolkit. By generating reusable Python libraries instead of isolated scripts, the system achieves a 30x speedup over traditional Claude Code baselines on complex data queries. This methodology demonstrates that smaller, faster models can outperform larger systems when equipped with pre-built domain tools.
A Three-Phase Architecture for Data Analysis
The system, known as NVIDIA KGMON (NeMo Agent Toolkit) Data Explorer, addresses the challenge of multi-step reasoning across structured, tabular datasets. Instead of executing scripts from scratch for every question, the framework divides data exploration into three distinct phases: Learning, Fast Inference, and Offline Reflection.
During the Learning phase, a heavyweight model analyzes representative tasks using Python interpreters and file detectors to build a centralized library of reusable functions named helper.py. In the Fast Inference phase, a lightweight model like Haiku 4.5 uses these pre-built tools to execute instructions in a single pass. Finally, an unsupervised Offline Reflection phase uses heavy models to audit code, evaluate consistency, and inject newly learned insights back into system prompts for future runs.
Benchmark Performance and Results
NVIDIA tested its architecture against the Data Agent Benchmark for Multi-step Reasoning (DABStep), a benchmark consisting of 450 financial payments sector tasks. While 16% of the tasks are simple single-dataset queries, 84% represent hard, multi-step tasks requiring rule interpretation and data cross-referencing.
According to research published by NVIDIA, the framework achieved the following benchmark scores:
- NVIDIA Data Explorer (Haiku 4.5): 89.95% on hard tasks, averaging 20 seconds per task and 1,870 characters of generated code.
- Claude Code baseline (Opus 4.5): 66.93% on hard tasks, averaging 10 minutes per task and 5,011 characters of generated code.
- DataPilot (AntGroup): 87.57% on hard tasks.
- DS-STAR (Google AI): 45.24% on hard tasks.
Architectural Trade-Offs and Considerations
While the three-phase approach dominated the hardest reasoning challenges, it comes with specific trade-offs. On basic single-dataset queries, the baseline Claude Code with Opus 4.5 slightly edged out NVIDIA's framework, scoring 90.2% compared to NVIDIA's 87.5% on Easy tasks.
Additionally, the system requires an upfront investment of computational resources during the initial Learning phase. Organizations must spend processing power running heavy models over training batches to build the function library before fast, lightweight inference can occur.