Feb 2024AI / ML1 min read

RAG Document Chatbot

Retrieval-augmented chatbot over PDF docs: embed with a hosted model, retrieve top chunks, answer with an LLM constrained to that context. Flask UI, and a clear path from one shared index to per-user indexes.

Grounding an LLM in a document set with LlamaIndex

Python / Flask / llama_index / RAG

What it does

Long PDFs are painful to search by hand. This app embeds a document set, retrieves the closest chunks for a question, and asks an LLM to answer only from that context.

From a question to a grounded answer
  1. 1Embed

    The PDF set is split into chunks and embedded once, when the app starts.

  2. 2Retrieve

    A question pulls back the chunks closest to it.

  3. 3Answer from context

    The LLM is asked to answer only from those chunks.

  4. 4Scale out

    One index over a fixed folder serves a shared corpus; per-user uploads move to per-session indexes or a vector database.

Design choices

  • The index is built once at startup over data/, which keeps answers fast for a shared corpus
  • Per-user uploads move to session-scoped indexes or an external vector database
  • Embeddings are built at startup rather than per request, so cold starts on serverless hosts are the cost to plan around

What it demonstrates

  • End-to-end RAG without hiding the architecture
  • Separating retrieval from generation so models/corpus can swap independently
  • Shipping a working UI, not only a notebook