ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

原文：ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

作者：Mobina Kashaniyan, Ali Jannesari

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Machine Learning

arXiv:2610.25519v1 (cs)

[Submitted on 18 Oct 2026]

Title:ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Authors:Mobina Kashaniyan, Ali Jannesari

View a PDF of the paper titled ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference, by Mobina Kashaniyan and 1 other authors

View PDF

HTML (experimental)

Abstract:Test-time scaling (TTS) improves large language model (LLM) reasoning by sampling multiple answers, but fixed sampling budgets ignore both query difficulty and the changing energy cost of inference under GPU contention. We measure up to a 4.1-fold increase in energy for the same four-sample query as GPU load rises. We present ATEST, a resource-aware TTS framework that jointly considers the expected value of additional samples and their current GPU cost. ATEST-Online, its default mode, requires no offline calibration: it incrementally generates samples, estimates the stability of the plurality answer, monitors live GPU conditions, and learns step energy cost online to decide when to stop. ATEST-Calibrated instead selects a per-query budget using offline accuracy and cost models, favoring deployments with stable loads. We evaluate both modes on four reasoning and question-answering datasets using Phi-3-mini on an NVIDIA A100 under controlled varying load, and test transfer to an NVIDIA L40S and Qwen2.5-3B. Across the four A100 datasets, ATEST-Online answers every query without exceeding the energy budget, while achieving the highest or statistically tied accuracy and reducing sampling as load increases. On GSM8K at high load, it achieves 60.0% accuracy with no deferrals, versus 44.0% accuracy and 42.4% deferrals for ATEST-Calibrated. Ablations show that online cost learning drives adaptation to load. Including offline profiling, ATEST-Online uses 42 times less GPU time and 48 times less energy than ATEST-Calibrated for the evaluated workload. Online control trades higher tail latency and lower throughput on short tasks for robustness under varying load; offline calibration remains competitive under stable conditions.

Comments:

12 pages, 4 figures, 8 tables

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF)

ACM classes:

C.4; D.4.8; I.2.7

Cite as:

arXiv:2610.25519 [cs.LG]

(or

arXiv:2610.25519v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.25519

Focus to learn more

arXiv-issued DOI via DataCite

主题

机器学习 · 人工智能 · 自然语言处理 · 分布式计算 · cs.PF

由「前沿雷达」于 2026-10-19 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。