Document Store: Pickle Metadata

Overview

Unlike the FAISS index that only stores math vectors, the document store acts as the “file cabinet” holding the actual human-readable text and the mapping relationships between Children and Parents.

Structure / Schema

The document store comprises two critical pickle files:

1. doc_store.pkl: Stores the Actual Text of the Parent Chunks.
2. child_nodes.pkl: Stores the Map mapping Child IDs to Parent IDs.

Flow / Behavior

After the FAISS vector index finds a matching Child vector:

graph TD
    Match["📍 Found Child Chunk #456"] --> Lookup["🗺️ Look up Parent ID in child_nodes.pkl"]
    Lookup --> Retrieve["📦 Retrieve Parent text from doc_store.pkl"]
    Retrieve --> AI["🤖 Feed Parent Text to AI"]

Related Components

• vector-store
• system-architecture

---

Last Updated: 2026-05-01