LlamaIndex is an MIT-licensed Python framework for connecting an LLM to your own documents. To get started, install it on Python 3.10 or newer with pip install llama-index, set OPENAI_API_KEY, put your files in a data/ folder, and run five lines of code: load, index, build a query engine, ask.
Those five lines hide the decisions that matter: which packages you actually installed, where the embeddings went, what happens on the second run, and why the answer is wrong when the source document plainly contains it. The framework overview confirms the MIT license, and the 0.14.25 release on PyPI, published on 21 September 2026, requires Python 3.10 or later. The steps below follow the framework’s own starter example, then cover the three decisions that turn a demo into something you can run twice.
Getting started with LlamaIndex: the order to learn it
Each step depends on the one before it:
- Install the bundle plus the file readers on Python 3.10 or newer.
- Learn six nouns: Document, Node, index, retriever, query engine, chat engine.
- Run the five-line loop on a small folder and print
response.source_nodes. - Override the inherited defaults through
Settings, starting with the embedding model. - Persist the index so the second run does not re-embed anything.
- Move vectors into a dedicated store once more than one process needs them.
- Build a small evaluation set before you change chunk size, models or prompts.
Agents come after that. The official starter tutorial now opens with a FunctionAgent calling a multiply tool, and only later wraps a query engine as a search_documents tool. For a retrieval project the query engine is the part that decides answer quality, so learn it first.
Before you install anything
LlamaIndex describes itself as a framework for building LLM-powered agents over your data, and its own documentation breaks a RAG application into five stages: loading, indexing, storing, querying, and evaluation. Keep the first four in mind while you build, because every error you hit belongs to exactly one of them, and knowing which stage failed is most of the debugging.
You need three things to start:
- Python 3.10 or newer and a virtual environment.
- An API key for whatever model you intend to use. The starter example reads
OPENAI_API_KEYfrom the environment. A local model works too (see below). - A folder of documents. Anything the default reader can parse: text, Markdown, PDF, Word.
You do not need a LlamaIndex account. The hosted platform formerly called LlamaCloud has been renamed LlamaParse; it is a separate service for difficult documents such as scanned PDFs and slide decks, and the open-source framework runs without it.
The vocabulary you need first
Most beginner confusion is a naming problem. These are the terms the API reference assumes you know, as defined in the documents and nodes guide and the high-level concepts page:
| Term | What it is | Where you meet it |
|---|---|---|
| Document | A container around any data source: text plus metadata and relationships | Returned by SimpleDirectoryReader(...).load_data() |
| Node | A chunk of a Document; inherits its metadata, such as file_name | Created inside from_documents |
| Index | The structure that holds nodes for retrieval; a vector index stores an embedding per node | VectorStoreIndex |
| Retriever | Fetches the nodes closest to the question | Built for you by as_query_engine() |
| Query engine | Question in, answer plus the context it used out | index.as_query_engine() |
| Chat engine | The same flow across multiple turns | index.as_chat_engine() |
| Agent | An LLM with tools and memory, choosing the next tool in a loop | Wraps a query engine as one tool |
Install: one package or many
Since version 0.10 the project is not a single library. The ecosystem is a set of namespaced packages: llama-index-core holds the framework, and every model provider, vector store, and reader ships separately as llama-index-llms-openai, llama-index-vector-stores-qdrant, and so on. The plain llama-index name is a starter bundle. The installation page still documents it as core plus llama-index-llms-openai, llama-index-embeddings-openai and llama-index-readers-file, but the bundle’s pyproject.toml for 0.14.25 depends only on core, the two OpenAI packages and nltk. The file readers no longer come with it.
That gap bites on the first run. Without llama-index-readers-file, SimpleDirectoryReader logs a warning and loads no default readers, so a .pdf or .docx falls through to a plain-text decode of its raw bytes. You get Documents full of noise and no exception. Install the readers explicitly:
pip install llama-index llama-index-readers-file
export OPENAI_API_KEY=sk-...Install the bundle to learn the shape of the API. Move to llama-index-core plus explicit integrations when you care about what is in your image; a production service usually pins core plus only the integrations it actually imports.
The five lines
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What does this corpus say about X?")
print(response)That is the whole loop, and it maps one-to-one onto the documented ingestion, indexing and retrieval stages.
SimpleDirectoryReader is the loading stage. It walks the folder, picks a parser per file extension, and returns Document objects. Pass recursive=True to read subfolders and required_exts=[".md", ".pdf"] to skip everything else, both documented in the SimpleDirectoryReader guide. Print the length of documents and inspect the extracted text before indexing. For scanned PDFs, check whether the selected parser extracted usable text. Diagnose missing documents separately from retrieved passages that do not answer the question.
VectorStoreIndex.from_documents does the indexing stage, and it does more than the name suggests. It splits each document into nodes with the default sentence splitter, sends every node to an embedding model, and stores the vectors. This is the step that costs money and time. In the starter configuration the embeddings come from OpenAI and the vectors live in memory.
as_query_engine() builds the querying stage: a retriever, an optional set of node postprocessors, and a response synthesizer. Calling it with no arguments accepts every default, including a retriever that returns only the top two nodes.
query() embeds your question and retrieves the closest nodes. When nodes are available for synthesis, it packs them into a prompt with the question and returns the model’s answer along with the source nodes it used. Always look at response.source_nodes while you are learning. It is the difference between “the model is bad” and “the retriever handed the model the wrong paragraph”, and those have completely different fixes.
The defaults you inherit
Anything you do not set comes from a default, and these are the ones that surprise beginners:
| Setting | Default | What to do |
|---|---|---|
| LLM | OpenAI gpt-3.5-turbo | Set Settings.llm; the starter itself uses gpt-4o-mini |
| Embedding model | text-embedding-ada-002 | Docs recommend text-embedding-3-small for new projects |
| Chunk size | 1,024 tokens | Tune per corpus against an evaluation set |
similarity_top_k | 2 nodes | Raise it when answers span several sections |
| Vector store | In memory, inside the Python process | Persist it, or use a dedicated store |
The model defaults come from the starter tutorial, the chunk size and top-k values from the core constants file. The embedding default is the costly one to leave alone. On OpenAI’s pricing page, checked in October 2026, text-embedding-ada-002 costs $0.10 per million tokens against $0.02 for text-embedding-3-small, and whichever model builds the index must also embed every query, so switching later means re-embedding the whole corpus. For chunk size, run the sweep in the chunk size guide instead of guessing.
Configure once, with Settings
The first thing most people want to change is the model. The current way is the global Settings object:
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.chunk_size = 512If you find a tutorial that builds a ServiceContext and passes it around, that tutorial predates version 0.10. ServiceContext was deprecated in favour of Settings in 0.10 and removed in 0.11, so the code will not run on a current install. This is the single most common reason a copied snippet fails with an import error, and the migration guide in the sources exists specifically for it.
Settings attributes are lazily instantiated, so setting a value you never use costs nothing. Per-call overrides still work: anything you pass directly to a query engine or index beats the global.
Persist the index, or pay twice
The five-line version rebuilds the entire index on every run. Embedding is the slow, metered part of the pipeline, and re-embedding an unchanged corpus at every process start is pure waste.
index.storage_context.persist("storage")and on the next run:
from llama_index.core import StorageContext, load_index_from_storage
storage_context = StorageContext.from_defaults(persist_dir="storage")
index = load_index_from_storage(storage_context)Wrap those in a check for whether storage/ exists and the second run becomes near-instant. Two rules go with this. Record which embedding model produced the index, because vectors from different models are not comparable and a swapped model silently returns nonsense. And treat the persisted directory as a build artifact tied to a corpus version, not as a database you edit by hand.
Running LlamaIndex without an OpenAI key
The same pipeline runs offline once both models are local. The local-model starter pairs Ollama for generation with a Hugging Face embedding model, and says you need a machine with roughly 32GB of RAM.
pip install llama-index-core llama-index-readers-file llama-index-llms-ollama llama-index-embeddings-huggingface
ollama pull llama3.1from llama_index.core import Settings
from llama_index.llms.ollama import Ollama
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")
Settings.llm = Ollama(model="llama3.1", request_timeout=360.0, context_window=8000)The five lines do not change. request_timeout=360.0 gives slow local generation six minutes, and the tutorial sets context_window=8000 by hand to limit memory use. When you reload a persisted index, pass the same embed_model that built it: vectors from a local model and a hosted one are not interchangeable.
Moving to a real vector store
The default store keeps vectors in the Python process. That is correct for a prototype and wrong for anything with more than one worker. A dedicated vector store separates the index lifecycle from the app lifecycle, and Qdrant is the usual first step in this stack:
pip install llama-index-vector-stores-qdrantimport qdrant_client
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.vector_stores.qdrant import QdrantVectorStore
client = qdrant_client.QdrantClient("<qdrant-url>", api_key="<qdrant-api-key>")
vector_store = QdrantVectorStore(client=client, collection_name="documents")
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)The application code above this line does not change. That is the argument for the abstraction: the retriever, postprocessors, and query engine are unaware of which store is underneath.
Before adopting this backend, use VectorOps Report’s Qdrant vs Weaviate vs Milvus comparison to review the database options. Its guide to how to choose a vector database covers production selection criteria beyond this integration example.
Before you provision anything, it helps to know roughly how large the index will be. Chunk count scales with corpus size divided by chunk size, and memory scales with chunk count times embedding dimensions times four bytes, plus graph overhead for an approximate index. The site’s chunking and vector store sizer does that arithmetic so you can compare a 384-dimension local embedding model against a 1,536-dimension hosted one before committing to an instance size.
Query engine or chat engine
as_query_engine() answers one question with no memory of the last one. That is the right default for search-style traffic and for anything you evaluate, because each call is independent and reproducible.
as_chat_engine() adds conversation state. Its retrieval behavior depends on chat_mode: condense_question rewrites a follow-up using chat history before calling a query engine, while context retrieves using the current message. The simple mode does not query the index. Compare query engine vs chat engine for the mode choices and the evaluation implications.
Start with a query engine. Add chat only when the product genuinely needs multi-turn, and evaluate both separately.
The five errors everyone hits first
| Error | What it means | Fix |
|---|---|---|
ImportError on ServiceContext | Snippet predates version 0.10 | Use the Settings object instead |
ModuleNotFoundError on a vector store or LLM import | Integration package not installed | Install the specific llama-index-* package |
Warning that llama-index-readers-file was not found; PDF text is garbage | File readers missing from the bundle | pip install llama-index-readers-file and reindex |
| No source nodes returned | Loading or retrieval needs inspection | Check len(documents), indexed node content, and retrieval filters |
| Startup slow on every run | Index rebuilt each time | Persist and load from storage/ |
Check loading, packaging, and retrieval before changing the model.
For a pipeline that runs but returns the wrong evidence, follow LlamaIndex retrieval troubleshooting to inspect source nodes before changing the model or prompt.
Evaluate before you believe it
A RAG pipeline that answers fluently and wrongly looks identical to one that works. LlamaIndex ships evaluation modules for both halves of the problem: retrieval metrics such as hit rate and mean reciprocal rank for whether the right node came back, and response evaluators for faithfulness, context relevancy, answer relevancy, and correctness for whether the answer used it properly. Many of these do not require ground-truth labels, and the framework can synthetically generate question and context pairs from your own text, so there is no excuse for shipping with zero measurements.
Build the evaluation set on day one, while the corpus is small. It is the only way to tell whether a change to chunk size helped.
What to read next
- The concepts behind the five lines, in one page: LlamaIndex ingestion, indexing and retrieval explained.
- Picking the framework at all: LlamaIndex vs LangChain, which to use for RAG.
- When the pipeline runs but the answers are wrong: LlamaIndex retrieval troubleshooting.
FAQ
is llamaindex good for beginners
Yes, if you already write basic Python. The high-level API goes from a folder of files to answers in five lines, and every default can be overridden later through Settings. What trips beginners up is packaging and defaults: install the file readers separately, set the embedding model explicitly, and read the source nodes before blaming the LLM.
can i use llamaindex without openai
Yes. OpenAI is only the default provider. Install the Ollama and Hugging Face integration packages, point Settings.llm at a local model such as Llama 3.1 and Settings.embed_model at an embedding model such as BAAI/bge-base-en-v1.5, and the indexing and query code stays identical. Budget around 32GB of RAM for the documented local setup.
what python version does llamaindex need
Python 3.10 or newer. Version 0.14.25, released on 21 September 2026 and still the latest on PyPI in October 2026, requires at least 3.10 and below 4.0. On an older interpreter pip skips releases it cannot install, so create the virtual environment with 3.10 or later before running pip install llama-index.
is llamaindex free
The framework is free and open source under the MIT license; you pay only for the model APIs it calls, or nothing beyond hardware with local models. On OpenAI’s October 2026 price list, text-embedding-3-small costs $0.02 per million tokens and gpt-4o-mini $0.15 per million input tokens. LlamaParse, the hosted parsing service formerly called LlamaCloud, is separate.
