Chunking
Also called Document chunking, Text splitting
Splitting documents into smaller passages before indexing them, so a search system can find and return just the relevant part.
What is chunking?
Chunking is the step in a RAG system where long documents are cut into smaller pieces, called chunks, before they're stored for search. When someone asks a question, the system retrieves the few chunks that match it and adds them to the AI model's prompt, instead of the whole document.
How does it work?
Each chunk is turned into an embedding by an embedding model and stored in a vector database. Common ways to split:
- Fixed size: every few hundred tokens, often with some overlap so a sentence isn't cut in half between two chunks.
- By structure: along headings, paragraphs, list items or table rows, which keeps related ideas together.
- By meaning: starting a new chunk where the topic changes.
Many systems also attach the document's title, section heading or a one-line summary to each chunk, so a passage still makes sense on its own.
What is it commonly confused with?
- Tokenization. A tokenizer splits text into the tiny units a model reads. Chunking splits documents into passages for search.
- Summarising. Chunks keep the original text word for word; nothing is condensed.
Why does it matter?
Chunking quietly decides how good a RAG system can be. Chunks that are too small lose their context ("it increased by 12%": what did?). Chunks that are too large bury the useful sentence and waste room in the context window. There's no single best size, so test with real questions and adjust.
