Knowledge Graph RAG (GraphRAG): What It Is and When It Beats Vector Search
How GraphRAG works, the published evidence for when graph retrieval outperforms vector similarity, and how it differs from a memory layer.
TL;DR
Knowledge graph RAG, often called GraphRAG, replaces or augments the vector-similarity step in retrieval augmented generation with traversal over a graph of entities and relationships. Instead of fetching the chunks that most resemble the question, the system finds the entities the question is about and follows edges to related facts. It wins decisively on questions that require connecting several hops of information and on questions about a whole corpus rather than a passage. It costs more to build, and for simple lookup questions plain vector search is faster and cheaper. This guide covers how it works, the evidence for when it helps, and how it differs from a memory layer.
How does knowledge graph RAG actually work?
Standard RAG embeds your documents as vectors, embeds the question, and returns the chunks with the closest vectors. GraphRAG adds a structured layer, typically in four stages:
1. Extraction. A model reads the source documents and pulls out entities (people, systems, customers, decisions) and the relationships between them.
2. Graph construction. Those entities become nodes and the relationships become edges, usually with the source passage retained as provenance on each edge.
3. Community detection. Clustering algorithms group densely connected nodes, and the system generates a summary for each cluster. Microsoft Research's GraphRAG implementation uses the Leiden algorithm for this step.
4. Query time. For a local question the system finds the relevant entities and traverses outward. For a global question it reads the cluster summaries rather than the underlying text.
The fourth stage is the one that explains the performance difference. A question like "what are the main themes across these two thousand support tickets" has no single passage that answers it, so vector search retrieves an arbitrary handful of tickets and the model generalises from a biased sample. Traversing pre-computed community summaries covers the corpus.
When does GraphRAG beat plain vector search?
Microsoft Research's published evaluation of GraphRAG reported substantial gains in comprehensiveness and diversity on global sensemaking questions over a private dataset, and noted that the advantage narrows on simple local questions where a single passage contains the answer. Read those results as directional rather than universal, because the evaluation used LLM-as-judge scoring on specific corpora, and results move with the corpus.
The pattern that holds across implementations:
1. Multi-hop questions favour graphs. "Which customers were affected by the outage that the payments migration caused" requires joining three facts that never appear in one paragraph. Vector search retrieves passages about each separately and hopes the model connects them.
2. Whole-corpus questions favour graphs strongly. Summarisation, theme extraction and "what changed" questions have no nearest neighbour to find.
3. Single-fact lookup favours vectors. If the answer sits in one chunk, embedding similarity finds it in one step for a fraction of the cost.
4. Freshly changing corpora punish graphs. Rebuilding entity extraction and community summaries after every update is expensive, which is why most implementations rebuild in batches and accept staleness in between.
GraphRAG, vector RAG and memory layers compared
| Dimension | Sentra | GraphRAG | Vector RAG |
|---|---|---|---|
| Answers multi-hop questions | Yes | Yes | Weakly |
| Answers whole-corpus questions | Yes | Yes | No |
| Single-fact lookup | Yes | Yes, higher cost | Yes, cheapest |
| Knows when a fact stopped being true | Yes, bi-temporal | No | No |
| Resolves contradictions between sources | Yes | Surfaces both | Returns both, silently |
| Ingest cost | Higher, continuous | High, batch rebuild | Lowest |
| Governed, role-scoped answers | Yes | Depends on implementation | Rarely |
The row that matters most for enterprise use is the third from the bottom. A knowledge graph records that two sources disagree. It does not, on its own, decide which is current, because a plain graph has no notion of time beyond what you model manually. That is the specific gap a bi-temporal memory layer fills: every fact carries both when it was true in the world and when the system learned it, so superseded facts can be retired rather than returned alongside their replacements.
What does it cost to build?
Three costs that teams underestimate, in order of how often they cause a project to stall:
1. Extraction is an LLM pass over the entire corpus. Every document is read by a model to pull entities. For a large corpus this is the dominant build cost and it recurs whenever the extraction prompt changes.
2. Community summarisation is a second full pass. Summaries are generated per cluster, and clusters change as the graph grows.
3. Schema decisions are hard to reverse. Choosing entity types and relationship types early determines what questions are answerable later, and rebuilding after a schema change means paying the extraction cost again.
A reasonable sequencing for teams evaluating this: start with vector RAG, measure which failing questions are multi-hop or global, and only build a graph if that class is a meaningful share of real traffic. Building a graph for a corpus that is mostly answered by single-passage lookup is the most common wasted quarter in this area.
FAQ
Is GraphRAG the same as a knowledge graph?
Do I need a graph database to do GraphRAG?
Can I combine GraphRAG with vector search?
Does GraphRAG eliminate hallucination?
How is this different from an organizational memory layer?
The decision rule
Build a knowledge graph when a meaningful share of your real questions require connecting facts that never co-occur in one passage. Stay on vector search when they do not. Add a memory layer on top when the problem is not finding the connection but knowing which version of it is still true.