Comparative Study of Text Chunking Techniques

1. Introduction

With the practical adoption of LLMs in recent years, RAG (Retrieval-Augmented Generation) architectures that retrieve and reference external knowledge while generating responses have become widely adopted. RAG performance heavily depends on retrieval accuracy, which is determined by the quality of text chunks stored in the index.

Chunking refers to the process of splitting lengthy documents into semantically appropriate units. The quality of this segmentation directly affects precision and recall during retrieval, and consequently the accuracy and comprehensiveness of LLM-generated responses. Despite this importance, systematic comparative studies of chunking methods remain insufficient, and practitioners often rely on rules of thumb for method selection.

This paper surveys representative chunking methods, compares their characteristics, and organizes findings related to chunk size optimization. The findings presented here are derived from the author's research and development activities; specific implementation details are omitted.

2. Overview and Comparison of Chunking Methods

Text chunking methods can be broadly classified into the following five categories based on their segmentation criteria. The overview and characteristics of each method are described below.

2.1 Fixed-size Chunking

The most basic method, which mechanically splits text at a predetermined character or token count. Its advantages are extremely simple implementation and fast processing speed. However, it has a fundamental drawback: splits can occur mid-sentence or mid-paragraph, easily compromising semantic coherence.

2.2 Fixed-size with Overlap

An improvement on fixed-size chunking that introduces overlap regions between adjacent chunks. The overlap preserves contextual information around chunk boundaries, tending to improve retrieval accuracy compared to plain fixed-size chunking. However, there is a trade-off in increased index size due to overlapping portions. Typically, overlap is set at 10-20% of the chunk size.

2.3 Semantic Chunking

A method that uses embedding models to calculate semantic similarity between sentences or paragraphs, splitting at semantic boundaries. Cosine similarity between embedding vectors of adjacent text fragments is computed, and points where similarity falls below a threshold are designated as chunk boundaries. Since semantically coherent chunks are generated, improved retrieval accuracy can be expected. However, the embedding computation cost significantly increases processing time compared to fixed-size methods.

2.4 Recursive Chunking

A method that applies multiple delimiters in order of priority, splitting text in stages. It first attempts to split by paragraph breaks (double newlines); if resulting chunks exceed the target size, it then splits by sentence boundaries (periods), and if necessary, by word boundaries, recursively applying finer granularity. This balanced method respects the natural structure of documents while keeping chunk sizes within a target range.

2.5 Document Structure-based Chunking

In structured documents such as Markdown, HTML, or PDF, this method uses document structure elements like headings, sections, and lists as splitting criteria. Since segmentation follows the logical structure of the document, each chunk has high semantic completeness. However, it is difficult to apply when input documents lack clear structure (e.g., plain text), and it depends on the document format.

2.6 Method Comparison

Table 1 presents a comparison of the above methods based on key evaluation criteria.

As the comparison table clearly shows, each method has inherent trade-offs, and no single universal method exists. Semantic chunking is advantageous when semantic coherence is prioritized, but increased processing costs are unavoidable. Recursive chunking is a practical choice when seeking a balance between processing speed and semantic quality.

3. Considerations on Optimal Chunk Size

Equally important to the choice of chunking method is chunk size, a parameter that affects both retrieval accuracy and generation quality, requiring careful consideration.

3.1 Impact of Chunk Size

Chunk size configuration can be understood as a problem of balancing the following two competing requirements.

3.2 Experimental Findings

In the author's research and development, RAG system performance was evaluated across various chunk sizes on text data from multiple domains. The key findings are reported below.

1. 256-512 tokens as a general recommended range: In many use cases, the 256-512 token range demonstrated the best balance between retrieval accuracy and generation quality. This range approximates the length of typical paragraphs, making it easier to maintain semantic completeness.
2. Optimal values vary by domain characteristics: It was confirmed that optimal chunk size differs depending on the document domain, such as technical documents, legal documents, or general articles. In domains where specialized terminology definitions and statutory citations are frequent, slightly larger chunk sizes (512-768 tokens) were effective.
3. Correlation with question granularity: It was observed that smaller chunks are advantageous for factoid-type questions (those asking for specific facts), while larger chunks are favorable for summary/explanatory questions.
4. Effect of overlap: Setting overlap equivalent to 10-15% of the chunk size was confirmed to mitigate information loss at chunk boundaries. However, overlap exceeding 20% showed performance degradation due to increased redundancy.

Note: The above findings are results from the author's experimental environment. Optimal values may vary depending on the embedding model, LLM, evaluation dataset, and other factors. Benchmark evaluation using target domain data is recommended for production deployment.

3.3 Framework for Chunk Size Selection

1. Set a baseline: Start with approximately 512 tokens as the baseline.
2. Analyze use cases: Analyze the expected question granularity and document characteristics, adjusting size as needed.
3. Apply overlap: Set overlap at 10-15% of the chunk size.
4. Evaluate and iterate: Use representative query sets to evaluate retrieval accuracy and generation quality, fine-tuning parameters accordingly.

4. Discussion and Summary

4.1 Method Selection Guidelines

When selecting a chunking method, it is essential to choose an appropriate approach based on system requirements and constraints.

4.2 Future Prospects

• Adaptive chunking: An approach that dynamically switches chunk size and splitting methods based on document content and structure.
• Hierarchical chunking: A method that generates chunks at multiple granularity levels (coarse and fine) and selects the appropriate granularity during retrieval.
• Multimodal support: Extension of chunking methods to documents containing not only text but also tables, figures, and mathematical formulas.
• LLM-powered chunking: A method that uses the LLM itself to estimate optimal chunk boundaries. High accuracy but with cost trade-offs.

4.3 Conclusion

This paper surveyed major text chunking methods, compared their characteristics, and reported findings on chunk size selection. Chunking is the "unsung hero" of RAG systems, and its optimization directly improves overall system quality. The author hopes this paper serves as a reference for practitioners formulating chunking strategies.