RAG over PDFs with LlamaIndex
How a PDF chatbot grounds answers with VectorStoreIndex — chunking, retrieval, generation — and why a global in-memory index is a demo ceiling, not a product architecture.
Retrieval-augmented generation keeps an LLM honest about a document it never trained on: embed the PDF, retrieve relevant chunks, answer from that context only.
Companion project: RAG Document Chatbot.
The happy path
documents = SimpleDirectoryReader("data/").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()Each query embeds the question, pulls top-k chunks, and passes them as context to the LLM. The model can still misread a chunk — it shouldn’t invent facts from outside the retrieval set.
Where demos stop
Building one global index at process start works for a fixed corpus. It fails when:
- Users upload different PDFs
- Sessions need isolation
- The host has no persistent filesystem (many serverless runtimes)
Fixes: session-scoped indexes, or a managed vector store (Chroma, Qdrant, Weaviate, etc.).
Bonus: keyword side-index
A KeywordTableIndex can surface “what topics does this doc cover?” without another LLM call — handy as a sidebar preview.
Takeaway
RAG skill is wiring ingestion → embed → retrieve → generate clearly. Production skill is scoping indexes and persistence. Ship the first to learn; don’t pretend the second is free.