原文:AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control
作者:Yiyang Du, Yuxuan Kuang, Qitao Zhao, Yixuan Wang, Zhanqiu Guo, Yi Xu, Xin Li, Wenbin He, Liu Ren, Chenyan Xiong
来源:arXiv cs.CL(自然语言)
正文
Computer Science > Robotics
arXiv:2611.26613v1 (cs)
[Submitted on 19 Nov 2026]
Title:AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control
Authors:Yiyang Du, Yuxuan Kuang, Qitao Zhao, Yixuan Wang, Zhanqiu Guo, Yi Xu, Xin Li, Wenbin He, Liu Ren, Chenyan Xiong
View a PDF of the paper titled AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control, by Yiyang Du and 9 other authors
View PDF
HTML (experimental)
Abstract:Frontier multimodal language agents such as GPT-6 Astra have demonstrated strong capabilities in controlling robots directly via low-level numerical actions without high-level primitives. However, existing benchmarks report only episode-level success rates, which show neither how these agents succeed nor where they fail. In this paper, we present a systematic action-level analysis of frontier agents on direct robot control across 12 manipulation benchmarks. We introduce AgentRoboCritic, an intent-grounded critique framework in which the policy agent states a local intent alongside each action, and a critic evaluates the action’s outcome and failure mode against this intent. Our action-level analysis demonstrates the promising capabilities of frontier agents on direct robot control, including closed-loop recovery from local execution failures, reliable semantic grounding across diverse scenes, and strong zero-shot manipulation without task-specific adaptation. It also identifies their current limitations, including task-relevant physical execution bottlenecks such as grasping and alignment, and uneven recovery across failure modes. Finally, we validate our critic on a 400-action human-annotated benchmark and release a dataset of 12,844 annotated actions alongside a fine-tuned open-source critic, Muse-Glimmer-30B, whose failure-mode accuracy rises from 32.58% to 74.24%, enabling low-cost action-level analysis.
Comments:
28 pages. Project page: this https URL
Subjects:
Robotics (cs.RO); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2611.26613 [cs.RO]
(or
arXiv:2611.26613v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.26613
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
主题
机器人 · 自然语言处理 · 计算机视觉
由「前沿雷达」于 2026-11-20 采集。正文取自原文页面,已保留出处链接。标题与正文版权归原作者所有。
评论
加载中…