Module 3 of 6 · 30 minutes

Preparing documents: chunking and metadata

Most RAG quality problems start before a single question is asked — in how documents are cleaned, split and labelled.

Learning objectives

Clean and de-duplicate source documents
Explain why chunking matters
Compare chunking strategies and implement one
Design useful chunk metadata
Plan index updates

3.1 Loading and cleaning

Real documents are messy: PDFs with headers and footers on every page, tables, scanned images, HTML menus, duplicate versions. Garbage in the index means garbage in the answers.

Extract text with tools suited to each format (PDF parsers, OCR for scans, HTML-to-text)
Remove boilerplate: navigation, repeated headers, footers, cookie banners
Keep structure that carries meaning: headings, lists, table rows
De-duplicate and keep only the current version of each document
Respect access rights — record who is allowed to see each document

3.2 Why chunk at all?

Embedding a whole 50-page document gives one vector that blurs every topic in it together. Retrieval works better when each vector represents one focused piece of content. Chunking splits documents into passages small enough to be specific, but large enough to make sense on their own.

3.3 Chunking strategies

Strategy	How	Pros / cons
Fixed size	Every N tokens or characters	Simple; may cut sentences and ideas in half
Fixed size with overlap	Each chunk repeats the end of the previous one	Reduces lost context at boundaries; more storage
Recursive / structure-aware	Split on headings, then paragraphs, then sentences	Keeps ideas together; a strong default
Semantic	Split where the topic changes (using embeddings)	Coherent chunks; slower to build
Document-specific	Rows for tables, functions for code, Q&A pairs for FAQs	Best quality for structured content

There's no universal best size. A few hundred tokens with some overlap is a common starting point — then test (Module 6).

Python · a simple paragraph-aware chunker with overlapdef chunk_text(text, max_words=180, overlap_words=40):
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks, current = [], []
    for para in paragraphs:
        words = para.split()
        if current and len(current) + len(words) > max_words:
            chunks.append(" ".join(current))
            current = current[-overlap_words:]          # carry some context forward
        current.extend(words)
    if current:
        chunks.append(" ".join(current))
    return chunks

Chunk size is a trade-off. Tiny chunks lose context; huge chunks mix topics. Mid-sized chunks with a little overlap are a good default — then test.

3.4 Metadata: the secret weapon

Store metadata with every chunk: source document, title, section heading, date, version, author, product, region and access permissions. Metadata lets you:

filter retrieval (only current policies; only UK documents; only what this user may see);
cite sources precisely in answers;
add context to chunks — e.g. prefixing each chunk with its document title and heading.

{"id": "returns-policy-v7#3",
 "text": "Damaged items can be returned within 30 days ...",
 "source": "Returns Policy v7", "section": "Damaged goods",
 "updated": "2026-06-01", "region": "UK", "audience": ["staff", "customers"]}

Each chunk travels with metadata. It powers filtering (including permissions), precise citations and added context.

3.5 Keeping the index fresh

Documents change. Plan for incremental updates: re-process changed documents, delete chunks from removed ones, and record the version indexed. Stale chunks are a common source of confidently wrong answers.

Key takeaways

Clean text, keep meaningful structure, de-duplicate and record permissions.
Chunks should be specific yet self-contained.
Structure-aware chunking with overlap is a strong default; test sizes.
Metadata enables filtering, citations and added context.
Stale chunks cause confident wrong answers — keep the index fresh.