RAG with Parent Document Retrieval

Standard RAG chunks documents into small pieces for precise retrieval, but then passes only those small chunks to the LLM as context. This creates a problem: the retrieved chunk may be accurate but lacks the surrounding context needed to generate a complete answer. Parent Document Retrieval solves this by indexing small chunks for search but returning the full parent document section as context to the LLM.

The technique uses two storage layers: a vector store containing small child chunks (200-400 tokens) for high-precision retrieval, and a document store containing large parent chunks (1000-2000 tokens) that provide rich context. When a child chunk matches the query, the retriever fetches its parent instead, giving the LLM more coherent and complete information.

The below example implements Parent Document Retrieval for a ShopMax India product manual, retrieving precise matches but providing full section context to the LLM.

It gives the following output,

Retrieved 1 parent chunk(s):
Samsung Galaxy S24 Setup Guide.
Section 1: Getting Started. Insert the SIM card using the
provided tool. Press the power button for 3 seconds to boot.
Follow the setup wizard to connect to WiFi and sign in to your
Google account. Section 2: Camera. The 200MP camera supports
8K video at 30fps. Use Pro mode for manual control. Night mode
activates automatically in low light conditions.

The small child chunk "Night mode activates automatically in low light conditions" matched the query precisely, but the LLM received the full camera section as context. This gives the model enough surrounding information to answer follow-up questions about the camera without requiring additional retrievals. Use Parent Document Retrieval for ShopMax product manuals, policy documents, and any content where answers require reading a full section rather than an isolated sentence.