Multimodal AI models can answer complex questions about data in images and documents with startling accuracy, but a new paper from Arizona State University, "ViTaB-A: Evaluating Multimodal Large Language Models on Visual Table Attribution," identifies a critical flaw: these models often can't cite where in a table their answer actually came from.
The Black Box Problem in Data Analysis
The paper introduces ViTaB-A, a benchmark for testing whether multimodal large language models (mLLMs) can point to the exact rows and columns that support an answer, rather than just producing the answer itself. For AI to be useful in professional settings like finance, research, or compliance, getting the right answer is only half the battle — users need to verify where the information came from. According to the paper, models tested frequently gave correct answers while failing to identify the specific cell-level evidence behind them, meaning a confidently correct answer can still come with a fabricated or wrong justification.
Putting Models to the Test
Researchers evaluated several mLLMs against structured data in three formats commonly used by professionals:
- Image-based tables: screenshots of financial reports or spreadsheets
- Markdown tables: common in technical documentation and code repositories
- JSON data: a standard format for data interchange in web applications
For each format, models were scored not just on whether they answered a question correctly but on whether they could cite the precise cell-level evidence supporting that answer. The paper reports this attribution skill is weak across the models tested, though the article's source does not provide model-by-model accuracy figures.
Why It Matters
As companies deploy multimodal AI for data analytics, ViTaB-A gives the field a concrete way to measure explainability rather than treating it as an afterthought. Without reliable source attribution, AI-generated data analysis remains risky for high-stakes use cases — and this benchmark gives developers a metric to target as they try to close the gap.