SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping

原文：SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping

作者：Haoyuan Wang, Shuai Peng, Yongkai Liao, Yuxiao Sun, Wei Luo, Xinxing Chen, Jian Huang

来源：arXiv cs.RO（机器人）

正文

Computer Science > Robotics

arXiv:2611.02794v1 (cs)

[Submitted on 2 Nov 2026]

Title:SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping

Authors:Haoyuan Wang, Shuai Peng, Yongkai Liao, Yuxiao Sun, Wei Luo, Xinxing Chen, Jian Huang

View a PDF of the paper titled SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping, by Haoyuan Wang and 6 other authors

View PDF

HTML (experimental)

Abstract:Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IMU, and structured text observations. We also collect Soma-Success, an independent grasp-outcome dataset with 1,000 trials (998 retained after quality control), to evaluate frozen-encoder transfer for post-hoc grasp outcome classification and early grasp success prediction. Built on Soma, SomaAlign learns multimodal representations in three stages: masked reconstruction estimates relative modality reliability; reliability-aware contrastive alignment uses pair-specific adaptive temperatures to adjust alignment sharpness; and conflict-aware semantic constraints improve intra-sample cross-modal consistency while preserving global feature geometry. SomaAlign achieves an average Recall@10 of 90.66% across 30 directed cross-modal retrieval tasks. The learned representations also support physical attribute recognition, multimodal question answering, cross-modal generation, and cross-dataset transfer to grasp outcome prediction. On Soma-Success, frozen SomaAlign encoders with lightweight probes achieve 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. Together, Soma, Soma-Success, and SomaAlign provide a data and representation-learning foundation for robust and transferable multimodal perception in soft robotic manipulation.

Comments:

18 pages, 9 figures

Subjects:

Robotics (cs.RO); Machine Learning (cs.LG)

Cite as:

arXiv:2611.02794 [cs.RO]

(or

arXiv:2611.02794v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2611.02794

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

机器人 · 机器学习

由「前沿雷达」于 2026-11-02 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。