作者：Yuhai Long, Yuanxin Wei, Kai Wu, Jinhui Wei, Dan Huang, Jiangsu Du
来源：arXiv cs.DC（分布式并行）
正文
Computer Science > Distributed, Parallel, and Cluster Computing
arXiv:2610.03394v1 (cs)
[Submitted on 2 Oct 2026]
Title:EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures
Authors:Yuhai Long (1), Yuanxin Wei (1), Kai Wu (2), Jinhui Wei (1), Dan Huang (1), Jiangsu Du (1) ((1) School of Computer Science and Engineering, Sun Yat-sen University, (2) China Mobile Internet Company Ltd.)
View a PDF of the paper titled EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures, by Yuhai Long (1) and 7 other authors
View PDF
HTML (experimental)
Abstract:Emerging multi-agent LLMs demand privacy-preserving edge deployment, yet current inference systems struggle with these collaborative workflows. Specifically, the memory-bound decode phase causes severe bus contention on unified memory architectures (UMA), paralyzing naive CPU-GPU co-execution. Furthermore, speculative decoding in multi-agent workloads faces extreme variance in drafting difficulty, alternating between complex reasoning and predictable structured generation. Compounded by frequent tool-induced stalls, this highly fragmented execution severely underutilizes hardware and defeats traditional static batching.
We present EdgeAgent, a cross-layer inference system explicitly co-designed for edge UMA and multi-agent workloads. At the micro-architectural level, it bypasses rigid graph-compiler constraints to enable zero-copy UMA-aware tensor parallelism, utilizing asymmetric memory layouts to fully saturate both CPU and GPU compute units. At the scheduling level, it dynamically allocates draft budgets based on real-time sequence predictability to bound bandwidth waste. Concurrently, an asynchronous suspend-and-yield mechanism actively evicts stalled agents, ensuring continuous hardware saturation during unpredictable tool invocations.
Extensive evaluations on an Apple M4 SoC demonstrate that the UMA-aware execution alone contributes a 1.29x speedup over batched speculative decoding. Adding the agent-aware scheduling lifts the full EdgeAgent system to a 1.77x speedup under extreme tool-use latencies.
Subjects:
Distributed, Parallel, and Cluster Computing (cs.DC); Multiagent Systems (cs.MA)
Cite as:
arXiv:2610.03394 [cs.DC]
(or
arXiv:2610.03394v1 [cs.DC] for this version)
https://doi.org/10.48550/arXiv.2610.03394
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
主题
分布式计算 · 多智能体
由「前沿雷达」于 2026-10-02 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。
评论
加载中…