UniWALM: Unified World Action Language Model for Embodied Robot Control

原文：UniWALM: Unified World Action Language Model for Embodied Robot Control

作者：Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Barreiros, Manikantan Nambi, Jeroen Vanbaar

来源：arXiv cs.AI（人工智能）

正文

Computer Science > Robotics

arXiv:2610.28626v1 (cs)

[Submitted on 20 Oct 2026]

Title:UniWALM: Unified World Action Language Model for Embodied Robot Control

Authors:Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Barreiros, Manikantan Nambi, Jeroen Vanbaar

View a PDF of the paper titled UniWALM: Unified World Action Language Model for Embodied Robot Control, by Rahul Sajnani and 11 other authors

View PDF

HTML (experimental)

Abstract:Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all three modalities inside a single mixture-of-transformers backbone whose per-modality diffusion heads co-denoise synchronously, decoding subtask and textual memory in the same pass as action with no separate planning stage; the generated language is re-consumed on the next step, closing a language-as-memory loop without an external planner or memory bank. To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost. UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0’s keyframe-equipped setup (28.5%) using only a single anchor frame. On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).

Comments:

Preprint: 25 pages

Subjects:

Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

Cite as:

arXiv:2610.28626 [cs.RO]

(or

arXiv:2610.28626v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2610.28626

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

机器人 · 人工智能 · 计算机视觉

由「前沿雷达」于 2026-10-21 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。