Nereus: Adaptive Parallelism for LLM Post-Training

原文：Nereus: Adaptive Parallelism for LLM Post-Training

作者：Songlin Jiang, Tuo Shi, Sitong Zhang, Zeke Wang, Mario Di Francesco, Bo Zhao

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Distributed, Parallel, and Cluster Computing

arXiv:2609.34645v1 (cs)

[Submitted on 28 Sep 2026]

Title:Nereus: Adaptive Parallelism for LLM Post-Training

Authors:Songlin Jiang, Tuo Shi, Sitong Zhang, Zeke Wang, Mario Di Francesco, Bo Zhao

View a PDF of the paper titled Nereus: Adaptive Parallelism for LLM Post-Training, by Songlin Jiang and 5 other authors

View PDF

HTML (experimental)

Abstract:Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job’s distributed state, and coordinating GPU transfers across models and stages.

Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27$\times$ over OpenRLHF and by 1.10–1.47$\times$ over Verl across diverse clusters.

Subjects:

Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)

ACM classes:

C.2.4; C.1.4; I.2.6

Cite as:

arXiv:2609.34645 [cs.DC]

(or

arXiv:2609.34645v1 [cs.DC] for this version)

https://doi.org/10.48550/arXiv.2609.34645

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

分布式计算 · 人工智能

由「前沿雷达」于 2026-09-28 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。