Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

原文：Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

作者：Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi

来源：arXiv cs.AI（人工智能）

正文

Computer Science > Robotics

arXiv:2610.02204v1 (cs)

[Submitted on 1 Oct 2026]

Title:Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Authors:Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi

View a PDF of the paper titled Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents, by Yen-Jen Wang and 9 other authors

View PDF

HTML (experimental)

Abstract:Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation. During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures. It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses. Cross-task evaluation tests individual candidate changes and merged revisions before they are retained for reuse. At test time, a multimodal LLM uses the resulting system prompt and skill library to coordinate perception and robot control. On held-out initializations of 22 manipulation tasks, RPG improves task success from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 powered by GPT-6 Astra Pro (60.0%). After a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, with ten trials on each of three tasks. Project Website: this https URL

Comments:

17 pages, 6 figures, 10 tables

Subjects:

Robotics (cs.RO); Artificial Intelligence (cs.AI); Systems and Control (eess.SY)

Cite as:

arXiv:2610.02204 [cs.RO]

(or

arXiv:2610.02204v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2610.02204

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

机器人 · 人工智能 · eess.SY

由「前沿雷达」于 2026-10-02 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。