KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

原文：KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

作者：Soha Lee, Soojin Lee, Heesung Yang, Hyunju Song, Hyunji Lee, Jinsan An, Jeongwan Shin, Jin Hyun Park, Jun Lee, Hyeyoung Park, Kilim Nam

来源：arXiv cs.CL（自然语言）

正文

Computer Science > Computation and Language

arXiv:2609.19916v1 (cs)

[Submitted on 17 Sep 2026]

Title:KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

Authors:Soha Lee, Soojin Lee, Heesung Yang, Hyunju Song, Hyunji Lee, Jinsan An, Jeongwan Shin, Jin Hyun Park, Jun Lee, Hyeyoung Park, Kilim Nam

View a PDF of the paper titled KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms, by Soha Lee and 10 other authors

View PDF

HTML (experimental)

Abstract:Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging words and meanings. Existing Korean benchmarks are centered on established vocabulary and therefore provide limited coverage of such recent lexical change, and their English-oriented design makes it difficult to assess the typological properties of Korean, in which content words combine productively with functional morphemes. In this paper, we introduce KoNeoBench, a benchmark for evaluating LLMs’ understanding of Korean neologisms. KoNeoBench is built on 1,785 Korean neologisms attested in online news since 2020 and curated through expert lexicographic review. Each entry provides usage examples, word-formation analyses, and dictionary-style definitions. Based on this resource, we define four tasks and report results on recent models, together with a human baseline. Our experiments show that current LLMs exhibit clear limitations in recovering source components, distinguishing semantic categories, and generating accurate definitions. These results reveal specific aspects of recent Korean lexical change that remain challenging for current LLMs. KoNeoBench is available at this https URL .

Comments:

Accepted to Findings of EMNLP 2026. Code and data are available at the project repository

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as:

arXiv:2609.19916 [cs.CL]

(or

arXiv:2609.19916v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.19916

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

主题

自然语言处理 · 人工智能

由「前沿雷达」于 2026-09-20 采集。正文取自原文页面，已保留出处链接。标题与正文版权归原作者所有。