Context Is Not Free: Carbon-Aware LLM Routing for Multi-Turn LLM Web Services across Geo-Distributed Data Centers

原文：Context Is Not Free: Carbon-Aware LLM Routing for Multi-Turn LLM Web Services across Geo-Distributed Data Centers

作者：Xiaoyang Zhang, Yuejun Yan, Qingang Zhang, Yonggang Wen

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Distributed, Parallel, and Cluster Computing

arXiv:2610.35230v1 (cs)

[Submitted on 26 Oct 2026]

Title:Context Is Not Free: Carbon-Aware LLM Routing for Multi-Turn LLM Web Services across Geo-Distributed Data Centers

Authors:Xiaoyang Zhang, Yuejun Yan, Qingang Zhang, Yonggang Wen

View a PDF of the paper titled Context Is Not Free: Carbon-Aware LLM Routing for Multi-Turn LLM Web Services across Geo-Distributed Data Centers, by Xiaoyang Zhang and 3 other authors

View PDF

HTML (experimental)

Abstract:The sustainability of the Web has drawn growing attention, as large language model (LLM)-based web services leave a substantial carbon footprint. These services typically run in geo-distributed data centers whose grid carbon intensity varies across regions and over time, so routing each query to a cleaner region or a smaller model can reduce emissions. A conversation, however, is a stateful web session: each turn reuses the key–value (KV) cache of its history, which resides only where earlier turns were served. Serving a turn on another deployment requires recomputing this history, whose carbon cost grows with the conversation and can outweigh the saving of the move. Existing carbon-aware LLM routers treat a conversation’s queries as independent and ignore this cost. We present CarbonRouter, an online framework that jointly selects the model, GPU type, and region for each query to minimize carbon emissions. CarbonRouter introduces a cache-aware carbon cost model, which charges history recomputation to every candidate deployment using cached-token counts from the serving backends, and uses a lightweight gate that keeps a session on its cached deployment when it would likely move back at the next turn and rebuild its history again. We evaluate CarbonRouter in trace-driven experiments on two real-world conversation datasets with five LLMs, three GPU types, and four regions, where it achieves the lowest carbon of all policies. CarbonRouter reduces CO$_2$ by 16.1% on average relative to FairCarbon, a state-of-the-art carbon-aware router with the same action space, while also improving predicted answer quality and estimated latency. Moreover, the saving rises further to 23.1% for sessions longer than 30 turns.

Subjects:

Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Cite as:

arXiv:2610.35230 [cs.DC]

(or

arXiv:2610.35230v1 [cs.DC] for this version)

https://doi.org/10.48550/arXiv.2610.35230

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

分布式计算 · 人工智能 · cs.CY

由「前沿雷达」于 2026-10-26 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。