BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

原文：BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

作者：Chuxuan Hu, Yeye He, Penny Zhou, Wee Hyong Tok, Daniel Kang, Surajit Chaudhuri

来源：arXiv cs.DB（数据库）

正文

Computer Science > Machine Learning

arXiv:2609.20886v1 (cs)

[Submitted on 16 Sep 2026]

Title:BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

Authors:Chuxuan Hu, Yeye He, Penny Zhou, Wee Hyong Tok, Daniel Kang, Surajit Chaudhuri

View a PDF of the paper titled BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence, by Chuxuan Hu and 5 other authors

View PDF

HTML (experimental)

Abstract:Business intelligence (BI) is a cornerstone of enterprise decision-making and is widely used by enterprise users in software such as Power BI and Tableau. In traditional BI workflows, users need to prepare data by (1) identifying relevant tables, (2) performing data transformations, and (3) building join relationships, before they can (4) answer their business questions. These steps can be complex and time-consuming, making BI challenging.

Given the strong capabilities of large language models (LLMs) in working with data, we study their ability to answer BI questions end-to-end, without requiring users to manually perform the tedious preparation steps. To do this, we harvest a large collection of real-world BI projects from public sources, and manually extract pairs of (questions, ground-truth answers) from real user dashboards. The resulting benchmark, BI-Bench, is the first benchmark to systematically study LLMs’ ability on end-to-end BI.

We find that even frontier LLMs perform poorly on BI-Bench, with less than 50% accuracy. To address their limitations, we design a tool-augmented BI-Agent that decomposes BI workflows into subtasks on structured data, such as search, join, and transform, and orchestrates specialized data management methods across BI stages. Furthermore, we develop a post-training framework that synthesizes training trajectories from real BI projects, enabling BI-Agent to be further post-trained using both supervised fine-tuning (SFT) and reinforcement learning (RL). BI-Agent achieves substantial accuracy gains of up to 40 percentage points with vanilla LLMs, and post-trained BI-Agent yields gains of up to 30 points. Our results highlight the importance of combining tool-augmented reasoning with domain-specific post-training in complex BI workflows, and point to promising directions for future research.

Comments:

code and data are available at \url{this https URL}

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Databases (cs.DB)

Cite as:

arXiv:2609.20886 [cs.LG]

(or

arXiv:2609.20886v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2609.20886

Focus to learn more

arXiv-issued DOI via DataCite

主题

机器学习 · 人工智能 · 自然语言处理 · 数据库

由「前沿雷达」于 2026-09-17 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。