DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception

原文：DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception

作者：Jinghua Hou, Zhe Liu, Hengshuang Zhao

来源：arXiv cs.AI（人工智能）

正文

Computer Science > Computer Vision and Pattern Recognition

arXiv:2610.12266v1 (cs)

[Submitted on 8 Oct 2026]

Title:DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception

Authors:Jinghua Hou, Zhe Liu, Hengshuang Zhao

View a PDF of the paper titled DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception, by Jinghua Hou and 2 other authors

View PDF

HTML (experimental)

Abstract:Multimodal large language models have made remarkable progress in bridging vision and language, facilitating various perception tasks essential for human-machine interaction, robotics, and autonomous driving. However, existing MLLM-based perception methods predominantly rely on text-based coordinate representation, which suffers from excessive token overhead, or fixed-range quantization, which suffers from range and precision constraints, especially for 3D domains with unbounded spatial range and high localization accuracy requirements. To address these challenges, we propose a dynamic vector decoding method named DVD, which unifies the representation of 2D and 3D perception tasks. Specifically, we first transform diverse perceptual representation (i.e., 2D bounding boxes, 2D masks, and 3D bounding boxes) into 1D vector sequences, which are then mapped to compact discrete tokens in the high-dimensional space. Then, a lightweight de-tokenizer enables seamless integration with MLLMs by decoding output tokens back to original 2D and 3D perceptual representations. Extensive experiments on 2D and 3D perception benchmarks including RefCOCO series, SUN-RGBD, KITTI, Hypersim, nuScenes demonstrate that DVD achieves superior performance in 2D and 3D tasks and reduces significantly the token overhead and inference latency. DVD provides an efficient and general framework for integrating perception capabilities into MLLMs, overcoming the inherent limitations of existing methods.

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Cite as:

arXiv:2610.12266 [cs.CV]

(or

arXiv:2610.12266v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2610.12266

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

计算机视觉 · 人工智能

由「前沿雷达」于 2026-10-09 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。