ServeTwin: A Benchmark-Validated Simulator for Distributed LLM Architecture Exploration

原文：ServeTwin: A Benchmark-Validated Simulator for Distributed LLM Architecture Exploration

作者：Sungjoon Park, Changue Jung, Kyungno Joo, Mincheol Kang, Jaehyung Ahn, Sehwan Lee, Sangjoon Kim

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Performance

arXiv:2610.02732v1 (cs)

[Submitted on 2 Oct 2026]

Title:ServeTwin: A Benchmark-Validated Simulator for Distributed LLM Architecture Exploration

Authors:Sungjoon Park, Changue Jung, Kyungno Joo, Mincheol Kang, Jaehyung Ahn, Sehwan Lee, Sangjoon Kim

View a PDF of the paper titled ServeTwin: A Benchmark-Validated Simulator for Distributed LLM Architecture Exploration, by Sungjoon Park and 5 other authors

View PDF

HTML (experimental)

Abstract:Evaluating distributed LLM serving designs on physical clusters is costly. Yet existing simulators provide only subsets of the capabilities needed for realistic design exploration: stateful closed-loop execution, timing prediction without profiling target hardware, and direct execution of unmodified serving benchmarks. We present ServeTwin, a closed-loop simulator that couples specification-driven analytical timing with a stateful serving loop. This coupling captures feedback among request completion, scheduling, queue state, and KV-cache evolution, including prefill-decode disaggregation and multi-turn agent workloads. ServeTwin avoids target-hardware operator profiling through iSTAGE, an analytical trace generator that derives per-iteration traces from model and engine specifications while representing ragged batches and discrete engine mechanisms such as CUDA-graph batch padding. It further decomposes execution time into component-owned throughput, scheduler, and runtime costs. This ownership lets unaffected parameters transfer across platforms and confines recalibration to changed hardware or software components. ServeTwin implements a vLLM-compatible interface and runs unmodified serving benchmarks. Against real deployments, it reproduces InferenceX’s steady-state throughput-interactivity frontier with a 3.6% mean error and predicts LMBenchmark’s multi-turn performance with a 9.9% error while tracking KV-cache evolution. Pre-silicon sweeps reveal that the preferred HBM bandwidth-capacity tradeoff reverses across workload states and that increasing concurrency shifts the bottleneck from memory bandwidth to scheduler and runtime overheads. Together, these capabilities enable practical exploration of distributed LLM serving systems before target hardware is available.

Subjects:

Performance (cs.PF); Distributed, Parallel, and Cluster Computing (cs.DC)

Cite as:

arXiv:2610.02732 [cs.PF]

(or

arXiv:2610.02732v1 [cs.PF] for this version)

https://doi.org/10.48550/arXiv.2610.02732

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

cs.PF · 分布式计算

由「前沿雷达」于 2026-10-02 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。