D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?

原文：D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?

作者：Daifeng Li, Huiqiang Jiang, Chengruidong Zhang, Wei Wu, Xudong Guo, Jianhong Tu, Jianwei Zhang, Binhang Yuan, Dayiheng Liu

来源：arXiv cs.DC（分布式并行）

正文

Computer Science > Machine Learning

arXiv:2610.03226v1 (cs)

[Submitted on 2 Oct 2026]

Title:D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?

Authors:Daifeng Li, Huiqiang Jiang, Chengruidong Zhang, Wei Wu, Xudong Guo, Jianhong Tu, Jianwei Zhang, Binhang Yuan, Dayiheng Liu

View a PDF of the paper titled D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?, by Daifeng Li and 8 other authors

View PDF

HTML (experimental)

Abstract:GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal how the gap relates to design discovery and implementation. We introduce D2K-Bench, a diagnostic benchmark of 26 tasks and 85 workloads that measures how effectively agents translate expert design guidance into efficient GPU kernels. The guidance covers L1: high-level algorithmic insights, L2: dataflow design, and L3: low-level optimization tricks, including dependencies among these levels. Pairwise runs with and without guidance share task descriptions, workloads, tools, hardware, and a 350-turn budget. Complementary assessments examine independently proposed designs and the design properties implemented in generated code. Across five models on NVIDIA B200 GPUs, guidance raises correctness over 130 model-task pairs from 93.1% to 98.5% and increases the Performance Score over all 26 tasks from 1.46 to 1.95. For the three frontier models with correct submissions on all 26 tasks in both runs (GPT-6-Astra, Claude-Opus-4.8, and GPT-5.6-Sol), geometric mean speedup increases from $1.69\times$ to $2.49\times$. Across all five models, the mean combined implementation score increases from 57 to 70 out of 100. These results show the value of expert design guidance while identifying design properties that remain unimplemented.

Comments:

30 pages, 4 figures

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)

Cite as:

arXiv:2610.03226 [cs.LG]

(or

arXiv:2610.03226v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.03226

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

机器学习 · 人工智能 · 分布式计算

由「前沿雷达」于 2026-10-02 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。