Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

原文：Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

作者：Qi Chen, Yunfei Chu, Haolin He, Yifan Yang, Zihan Liu, Yuxuan Wang, Ziyang Ma, Ruiyang Xu, Meng Gao, Yinsong Yan, Ling Wang, Hui Wang, Wen Huang, Yiheng Chen, Guanrou Yang, Qiuqiang Kong, Jin Xu, Xie Chen

来源：arXiv cs.CL（自然语言）

正文

Computer Science > Computation and Language

arXiv:2609.21392v1 (cs)

[Submitted on 18 Sep 2026]

Title:Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

Authors:Qi Chen, Yunfei Chu, Haolin He, Yifan Yang, Zihan Liu, Yuxuan Wang, Ziyang Ma, Ruiyang Xu, Meng Gao, Yinsong Yan, Ling Wang, Hui Wang, Wen Huang, Yiheng Chen, Guanrou Yang, Qiuqiang Kong, Jin Xu, Xie Chen

View a PDF of the paper titled Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction, by Qi Chen and 17 other authors

View PDF

HTML (experimental)

Abstract:Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather than carefully composed text prompts. However, existing benchmarks of interactive capabilities still focus primarily on response quality, leaving a more fundamental question underexplored: can a model correctly infer the user’s underlying demand from complex multimodal interaction? Real-world user demands are often underspecified in speech and must be inferred from multimodal cues and dialogue history. This inference is further complicated by ambiguous or disfluent expression and noisy acoustic environments. Conversely, request-like speech may not constitute a demand to the assistant, leading to false triggers. We establish Omni Demand Understanding (ODU) as a distinct multimodal contextual inference problem: given an interaction stream, a model must detect whether a user demand is present and infer intent from multimodal and conversational context. ODU evaluates this capability along five dimensions, covering both single-turn and multi-turn interactions. We construct ODU-Bench using a challenge-driven taxonomy, taxonomy-guided agentic video generation, and human-recorded interactions, followed by media-grounded annotation and human verification. We evaluate 14 native MLLMs. Even the strongest, Gemini 3.1 Pro, recovers only 44.7% of key information that must be inferred from visual, acoustic, or conversational context. Moreover, 11 of the 14 models exhibit false-trigger rates above 50% on non-demand scenarios. These results reveal a systematic capability gap in current MLLMs’ ability to infer contextual user demands. We hope ODU can establish the evaluation of a previously underexplored yet essential capability in multimodal interaction: correctly understanding user demands before generating an appropriate response.

Subjects:

Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

Cite as:

arXiv:2609.21392 [cs.CL]

(or

arXiv:2609.21392v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.21392

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

自然语言处理 · 计算机视觉 · cs.MM · cs.SD

由「前沿雷达」于 2026-09-18 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。