JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments

原文：JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments

作者：Zhixi Cai, Fucai Ke, Sukai Huang, Maria Garcia de la Banda, Peter J. Stuckey, Gholamreza Haffari, Hamid Rezatofighi

来源：arXiv cs.RO（机器人）

正文

Computer Science > Artificial Intelligence

arXiv:2609.35032v1 (cs)

[Submitted on 28 Sep 2026]

Title:JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments

Authors:Zhixi Cai, Fucai Ke, Sukai Huang, Maria Garcia de la Banda, Peter J. Stuckey, Gholamreza Haffari, Hamid Rezatofighi

View a PDF of the paper titled JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments, by Zhixi Cai and 6 other authors

View PDF

HTML (experimental)

Abstract:In complex embodied visual reasoning scenarios, an agent often has only a limited field of view, and the evidence needed to answer a question may be distributed across time, viewpoint, and interacting objects. A model may therefore give a plausible answer without ever observing the relevant object, time, or view that supports it. Current visual reasoning benchmarks largely evaluate passive observations and final answers, overlooking settings that require active reasoning and evidence acquisition. We introduce JRDB-AVR, a benchmark derived from existing real-world JRDB robotics data through a structured question-generation engine that turns this gap into an explicit evaluation: an embodied agentic system receives a visual reasoning question, requests bounded observations by timestamp and viewing angle, and is evaluated on both the final answer and the grounded visual evidence supporting it. The benchmark contains diverse questions over multiple real-world environments involving temporal search, viewpoint selection, and human-oriented compositional reasoning. We also introduce JRDB-AVR-Agent, a reference active reasoning agentic method that maintains an explicit observation-grounded graph-based world model and answers through solving. Experiments reveal a substantial gap between answer accuracy and evidence accuracy in current baselines, showing that current VLMs can produce unsupported correct answers and that active evidence-aware evaluation is necessary for embodied visual reasoning. Code and benchmark are available at this https URL.

Comments:

NeurIPS 2026

Subjects:

Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

Cite as:

arXiv:2609.35032 [cs.AI]

(or

arXiv:2609.35032v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2609.35032

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

人工智能 · 计算机视觉 · 机器人

由「前沿雷达」于 2026-09-28 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。