Broadcast, Don't Replicate: Eliminating Weight HBM in Datacenter LLM Serving

原文：Broadcast, Don’t Replicate: Eliminating Weight HBM in Datacenter LLM Serving

作者：Hannah Atmer, Yuan Yao, Thiemo Voigt, Stefanos Kaxiras

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Hardware Architecture

arXiv:2611.17097v1 (cs)

[Submitted on 12 Nov 2026]

Title:Broadcast, Don’t Replicate: Eliminating Weight HBM in Datacenter LLM Serving

Authors:Hannah Atmer, Yuan Yao, Thiemo Voigt, Stefanos Kaxiras

View a PDF of the paper titled Broadcast, Don’t Replicate: Eliminating Weight HBM in Datacenter LLM Serving, by Hannah Atmer and 3 other authors

View PDF

HTML (experimental)

Abstract:Large language model (LLM) inference at datacenter scale is fundamentally bottlenecked by memory capacity and bandwidth limits. Because model weights are read-only and identical across instances, conventional architectures that replicate these parameters across the local DRAM of tens of thousands of accelerators incur massive capital expenditures, significant static power dissipation, and redundant memory-fetch energy. A recent proposal to use optical fiber loops as delay-line memory alleviates these issues, but its recirculating loop must hold the whole model: loop length grows with model size, slack is fixed when the ring is built, and the layouts that serve modern Mixture of Experts (MoE) models efficiently do not fit in it.

We propose Optical Broadcast Memory (OBM), an open-ended hierarchical Tap-and-Amplify fiber architecture that generalizes and replaces the optical ring paradigm. A centralized memory server continuously broadcasts weights to distributed inference nodes that passively tap the data stream using receive-only co-packaged optics, delivering parameters directly to compute units without intermediate buffering. To overcome the limitations of continuous weight streaming, we use dynamically-sized slack to accommodate large and variable context lengths. We show how removing the restrictions of the optical ring allows OBM to efficiently serve MoE models. Evaluated at a datacenter scale of 80,000 accelerators, OBM eliminates petabytes of HBM while being competitive or better in throughput and energy per token than an HBM-based system.

Subjects:

Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC); Emerging Technologies (cs.ET); Networking and Internet Architecture (cs.NI)

Cite as:

arXiv:2611.17097 [cs.AR]

(or

arXiv:2611.17097v1 [cs.AR] for this version)

https://doi.org/10.48550/arXiv.2611.17097

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

cs.AR · 分布式计算 · cs.ET · cs.NI

由「前沿雷达」于 2026-11-13 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。