Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

原文：Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

作者：Hochan Son, Kyungdoe Han, Jaehan Koh, Xiaowu Dai, Wenlu Xu, Guang Cheng

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Artificial Intelligence

arXiv:2610.07782v1 (cs)

[Submitted on 6 Oct 2026]

Title:Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

Authors:Hochan Son, Kyungdoe Han, Jaehan Koh, Xiaowu Dai, Wenlu Xu, Guang Cheng

View a PDF of the paper titled Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell, by Hochan Son and 5 other authors

View PDF

HTML (experimental)

Abstract:Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache memory binds. Many such systems add a persistent tier storing and recalling reasoning traces, usually validated by an ablation reporting an accuracy gain. We measure both on one three-tier agent architecture. Decomposition delivers: peak KV working set of 14.3 MiB per query against 35.5 and 35.3 MiB for single-pass and retrieval-augmented baselines. The persistent tier does not: across eight controlled dataset pairs at n=100 per arm it costs +0.368 MiB [+0.167, +0.590] of peak cache and produces no detectable accuracy change (+0.015, 95% CI [-0.011, +0.046]). We argue the null is structural: single-question benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve. Reaching it took four measurement corrections – three inflating the apparent benefit, the fourth making an effect that size look resolvable – none visible in the results table. We give the conditions an agent-memory ablation must satisfy and detection procedures that need no knowledge of the specific defect.

Comments:

13 pages, 1 figure. Accepted as a poster at the Machine Learning for Systems Workshop, NeurIPS 2026

Subjects:

Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)

ACM classes:

C.4; I.2.7; I.2.11

Cite as:

arXiv:2610.07782 [cs.AI]

(or

arXiv:2610.07782v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2610.07782

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

人工智能 · 自然语言处理 · 分布式计算 · 机器学习

由「前沿雷达」于 2026-10-06 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。