Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

原文：Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

作者：Ziyue Luo, Jiaxuan Cai, Cedric LeDenmat, Srijith Nair, Fatemeh Nourzad, Rohith Krishnan, Qinhang Wu, Jifan Zhang, Sungjae Lee, Zhe Li, Peiwen Qiu, Siddharth Shah, Rishabh Sharma, Sundararajan Srinivasan, Yinglun Xia, Xue Zheng, Zidong Liu, Bicheng Ying, Kaushik Chowdhury, Gauri Joshi, Yingbin Liang,

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Distributed, Parallel, and Cluster Computing

arXiv:2610.32215v1 (cs)

[Submitted on 23 Oct 2026]

Title:Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

Authors:Ziyue Luo, Jiaxuan Cai, Cedric LeDenmat, Srijith Nair, Fatemeh Nourzad, Rohith Krishnan, Qinhang Wu, Jifan Zhang, Sungjae Lee, Zhe Li, Peiwen Qiu, Siddharth Shah, Rishabh Sharma, Sundararajan Srinivasan, Yinglun Xia, Xue Zheng, Zidong Liu, Bicheng Ying, Kaushik Chowdhury, Gauri Joshi, Yingbin Liang, Robert Nowak, Srinivasan Parthasarathy, Saurav Prakash, Balaraman Ravindran, Sanjay Shakkottai, Ness B.Shroff, Haibo Yang, Aylin Yener, Jia Liu

View a PDF of the paper titled Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead, by Ziyue Luo and 29 other authors

View PDF

HTML (experimental)

Abstract:The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer scale of these models and their training data makes centralized development increasingly hard to sustain, and at the same time shuts out smaller institutions that hold relevant private data but lack comparable infrastructure. In this paper, we organize our positions on distributed LLM training around the pretraining and fine-tuning paradigms, covering federated learning, split learning, and decentralized approaches, and we assess their readiness for geo-distributed settings with heterogeneous hardware and unreliable connectivity. We then present a {\em unified} system architecture built around a cloud-hosted parameter server that coordinates heterogeneous clients over outbound gRPC connections, reusing one communication, compression, and fault-tolerance stack for both pretraining and a family of fine-tuning methods. We validate the design with real geo-distributed experiments on GPT-2 Medium and Llama3-1B pretraining across multiple university sites, showing stable convergence under realistic conditions. Finally, we chart the road ahead, identifying open problems that span frontier model support, elastic fault tolerance, adaptive compression, federated alignment, privacy and trust, incentive design, and system–algorithm co-design, all of which must be addressed to make democratized LLM training a reality.

Subjects:

Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)

Cite as:

arXiv:2610.32215 [cs.DC]

(or

arXiv:2610.32215v1 [cs.DC] for this version)

https://doi.org/10.48550/arXiv.2610.32215

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

分布式计算 · 机器学习

由「前沿雷达」于 2026-10-23 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。