Data Optimization for LLMs: A Survey

2026-10-27 12:00 arXiv cs.AI Ou Wu 约 11 分钟读完

Large language models (LLMs) have attracted great attention in both academic and industrial communities in recent years. Numerous survey studies have covered various aspects of LLMs, such as learning strategies, applications, alignment, explainability, and evaluation. However, little headway has bee…

Authors: Ou Wu

View a PDF of the paper titled Data Optimization for LLMs: A Survey, by Ou Wu

View PDF

HTML (experimental)

Abstract: Large language models (LLMs) have attracted great attention in both academic and industrial communities in recent years. Numerous survey studies have covered various aspects of LLMs, such as learning strategies, applications, alignment, explainability, and evaluation. However, little headway has been made in reviewing data-related aspects for LLMs, despite data being recognized as a key factor contributing to the emergence of the LLMs era. A wide range of data optimization algorithms and techniques are employed in LLMs' training and application processes. These techniques often originate from diverse deep learning domains, with their theoretical inspirations and heuristic motivations appearing unrelated to each other. This study aims to develop a comprehensive framework for LLMs' data optimization techniques to enhance data utilization and efficiency. We first discuss data challenges in LLMs' training and inference. Second, we summarize data perception dimensions for evaluating LLMs' data properties. These dimensions primarily include quality, difficulty, diversity, and redundancy, laying a foundation for subsequent optimization. Third, existing optimization techniques are reviewed and categorized into four groups: data selection, augmentation, reweighting, and others. Fourth, we analyze interconnections between these categories. Finally, we identify key challenges and future directions for LLMs' data optimization compared with conventional deep learning approaches.

Comments:

Accepted for publication in ACM Transactions on Knowledge Discovery from Data (TKDD) at Sep. 30, 2026

Subjects:

Artificial Intelligence (cs.AI)

Cite as:

arXiv:2610.33405 [cs.AI]

(or

arXiv:2610.33405v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2610.33405

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ou Wu [ view email ]

[v1]

Sat, 24 Oct 2026 02:36:59 UTC (3,267 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Data Optimization for LLMs: A Survey, by Ou Wu

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.AI

< prev

next >

new

recent

| 2026-10

Change to browse by:

References & Citations

SciX

Google Scholar

Semantic Scholar

export BibTeX citation

BibTeX formatted citation

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer ( What is the Explorer? )

Connected Papers Toggle

Connected Papers ( What is Connected Papers? )

Litmaps Toggle

Litmaps ( What is Litmaps? )

scite.ai Toggle

scite Smart Citations ( What are Smart Citations? )

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv ( What is alphaXiv? )

Links to Code Toggle

CatalyzeX Code Finder for Papers ( What is CatalyzeX? )

DagsHub Toggle

DagsHub ( What is DagsHub? )

GotitPub Toggle

Gotit.pub ( What is GotitPub? )

Huggingface Toggle

Hugging Face ( What is Huggingface? )

ScienceCast Toggle

ScienceCast ( What is ScienceCast? )

Demos

Replicate Toggle

Replicate ( What is Replicate? )

Spaces Toggle

Hugging Face Spaces ( What is Spaces? )

Spaces Toggle

TXYZ.AI ( What is TXYZ.AI? )