跳到正文

AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

人工智能 0 次浏览

原文:AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

作者:Yiyang Du, Yuxuan Kuang, Qitao Zhao, Yixuan Wang, Zhanqiu Guo, Yi Xu, Xin Li, Wenbin He, Liu Ren, Chenyan Xiong

来源:arXiv cs.CL(自然语言)

正文

Computer Science > Robotics

arXiv:2611.26613v1 (cs)

[Submitted on 19 Nov 2026]

Title:AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

Authors:Yiyang Du, Yuxuan Kuang, Qitao Zhao, Yixuan Wang, Zhanqiu Guo, Yi Xu, Xin Li, Wenbin He, Liu Ren, Chenyan Xiong

View a PDF of the paper titled AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control, by Yiyang Du and 9 other authors

View PDF

HTML (experimental)

Abstract:Frontier multimodal language agents such as GPT-6 Astra have demonstrated strong capabilities in controlling robots directly via low-level numerical actions without high-level primitives. However, existing benchmarks report only episode-level success rates, which show neither how these agents succeed nor where they fail. In this paper, we present a systematic action-level analysis of frontier agents on direct robot control across 12 manipulation benchmarks. We introduce AgentRoboCritic, an intent-grounded critique framework in which the policy agent states a local intent alongside each action, and a critic evaluates the action’s outcome and failure mode against this intent. Our action-level analysis demonstrates the promising capabilities of frontier agents on direct robot control, including closed-loop recovery from local execution failures, reliable semantic grounding across diverse scenes, and strong zero-shot manipulation without task-specific adaptation. It also identifies their current limitations, including task-relevant physical execution bottlenecks such as grasping and alignment, and uneven recovery across failure modes. Finally, we validate our critic on a 400-action human-annotated benchmark and release a dataset of 12,844 annotated actions alongside a fine-tuned open-source critic, Muse-Glimmer-30B, whose failure-mode accuracy rises from 32.58% to 74.24%, enabling low-cost action-level analysis.

Comments:

28 pages. Project page: this https URL

Subjects:

Robotics (cs.RO); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

Cite as:

arXiv:2611.26613 [cs.RO]

(or

arXiv:2611.26613v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2611.26613

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

机器人 · 自然语言处理 · 计算机视觉


由「前沿雷达」于 2026-11-20 采集。正文取自原文页面,已保留出处链接。标题与正文版权归原作者所有。

评论

加载中…