Embedding Models Explained — And Why Embeddings Alone Are Not Memory

Every AI team eventually hits the same wall. You ship a RAG pipeline, wire it up to a vector store, and it works great in the demo. Then it goes into production and starts confidently retrieving the wrong version of a document, mixing up two customers with similar names, or surfacing a fact that was true six months ago but has since been overturned by a Slack thread, a Jira ticket, and an email nobody re-indexed. The embeddings didn't fail. They did exactly what embedding models are built to do: find things that look similar. The problem is that "looks similar" and "is true right now" are two completely different questions, and most AI memory systems only ever answer the first one.

This article breaks down what embedding models actually compute, where vector similarity earns its keep, and why every team that treats embeddings as a substitute for memory eventually has to rebuild their retrieval layer from scratch.

What an Embedding Model Actually Does

An embedding model is a neural network, usually a transformer encoder, that maps a piece of text (or an image, or audio) to a fixed-length vector of real numbers. A typical text embedding model outputs a vector of 384, 768, 1024, or 1536 dimensions depending on the model. OpenAI's text-embedding-3-large produces 3072-dimensional vectors by default, though it supports dimension truncation down to 256. Cohere's embed-v3 and open models like BAAI's bge-large or sentence-transformers' all-MiniLM-L6-v2 sit in the 384 to 1024 range.

The training objective is almost always contrastive: the model sees pairs or triplets of text (an anchor, a positive match, a set of negatives) and is optimized so that the cosine distance between the anchor and its positive is small, and the distance to negatives is large. This is why embeddings are good at capturing topical and lexical closeness. "The invoice was rejected due to a mismatched PO number" and "PO number mismatch caused the invoice to bounce" will land within a few degrees of each other in vector space, even though they share almost no exact wording.

What the model has no mechanism for is tracking whether either sentence is still true. Nothing in the training objective, the architecture, or the inference call encodes time, provenance, or update history. An embedding is a snapshot of meaning at the moment of encoding, computed independently of every other fact your system knows. That single limitation is the root of almost every production RAG failure teams report.

How Text Embeddings Turn Language Into Geometry

To use embeddings for retrieval, you chunk your source documents (commonly 256 to 1024 tokens per chunk, with overlap of 10 to 20%), run each chunk through the encoder, and store the resulting vector alongside metadata in a vector database (Pinecone, Weaviate, Qdrant, pgvector, etc.). At query time, you embed the user's question with the same model and search for the nearest chunk vectors, usually using cosine similarity or dot product.

Because brute-force nearest-neighbor search is O(n) per query, production systems use approximate nearest neighbor (ANN) indexes such as HNSW (Hierarchical Navigable Small World graphs) or IVF-PQ (inverted file with product quantization). These trade a small amount of recall, typically 1 to 5%, for orders-of-magnitude faster lookups, which is why a search over 10 million chunks can return in under 50ms.

This pipeline is genuinely powerful for one narrow task: finding text that is semantically related to a query when you don't know the exact keywords. It is a search mechanism, not a knowledge mechanism. It has no concept of which chunk is authoritative, which chunk is stale, or which two chunks contradict each other. It ranks by distance, not by correctness.

Semantic Search Embeddings: Strengths and Blind Spots

Semantic search embeddings solve the problem that keyword search (BM25, TF-IDF) cannot: matching intent when vocabulary differs. If a support ticket says "the sync job silently failed" and your knowledge base article says "background job did not raise an error but stopped processing," keyword search finds nothing. Embedding-based search finds it in the top three results.

But the same property that makes embeddings good at fuzzy matching makes them bad at precision-critical retrieval. Two customers named "Jon Park" at two different companies will produce nearly identical embeddings for sentences describing their support history, because the model is encoding surface semantics, not entity identity. A pricing document from Q1 and its superseding version from Q3 will often sit within 0.02 cosine distance of each other, because the wording barely changed even though the numbers did. Nothing in a standard vector index tells the retriever "prefer the Q3 version" or "these two Jons are different people." That resolution work either doesn't happen, or it gets bolted on as a fragile layer of metadata filters and rerankers applied after the fact, at query time, under latency pressure, using whatever context the retriever happens to have on hand.

Why Embeddings Alone Are Not Memory

Memory, for a person or a system, requires four things that embeddings do not provide on their own:

Identity resolution. Knowing that "Jon Park at Acme" and "J. Park, Acme Corp" and "the customer who emailed on March 3rd" all refer to the same entity. Embedding similarity approximates this sometimes and fails silently when it doesn't.

Temporal awareness. Knowing not just what was said, but when it was said, when it became true, and when it stopped being true. A vector store has no native concept of "valid from" or "superseded by." Every chunk is permanently present-tense.

Conflict resolution. Knowing that two retrieved chunks disagree, and having a rule for which one wins. Vector search will happily hand the model both the outdated policy and the corrected one in the same context window and let the LLM guess.

Update propagation. When a fact changes, memory needs to update once. With embeddings, updating a fact means re-chunking, re-embedding, and re-indexing the document it came from, and hoping every stale copy elsewhere in the store gets cleaned up too. In practice, stale vectors accumulate for months because nobody owns the deletion step.

Embeddings are a similarity function. Memory is a state machine. Bolting a similarity function onto raw documents and calling it "memory" is why RAG demos degrade the moment real-world data starts changing, which is to say, immediately.

The Retrieval-Time Guessing Problem

The deeper architectural issue is when the resolution work happens. A standard RAG pipeline defers every hard decision, which chunk is current, which entity is meant, which fact is authoritative, to query time. That is the worst possible moment to make that decision, for three reasons.

First, latency. Reranking, filtering, and disambiguation at query time add real cost to every single request, whereas resolving meaning once at write time means every subsequent query benefits from work already done.

Second, context. At query time you have the user's question and a pile of retrieved chunks. You do not have the full history of how a fact evolved, who asserted it, or what superseded it. At write time, when a document or message first enters the system, you have far more signal available to correctly place it in context.

Second (again, because it compounds), token cost. Query-time disambiguation usually means stuffing more candidate chunks into the context window so the model can "figure it out," along with instructions telling it to be careful about conflicting information. That inflates token usage on every single call, and it still produces wrong answers when the model guesses badly. Teams running this pattern at scale routinely see 2 to 4x the token spend of a system that resolves ambiguity before the query ever arrives.

This is the specific design choice Sentra makes differently. Instead of asking the retriever to guess the right answer under time pressure, Sentra resolves meaning once, at write time, as information enters the org-wide context graph. Entity resolution, temporal versioning, and conflict handling happen when a fact is written, not re-derived on every query. That is also why teams moving from pure vector RAG to Sentra's context graph report roughly 70% lower token spend: the model isn't being handed five redundant, partially-contradictory chunks and asked to sort it out anymore. It is handed the resolved fact.

Vector Embeddings vs. a Context Graph: Two Different Jobs

It helps to be precise about what each layer is actually for. Embeddings are a matching mechanism. A context graph is a memory mechanism. Most production systems need both, but they are not interchangeable, and treating the first as a substitute for the second is where teams get burned.

The verdict is not that vector search is bad. It is that vector search answers "what looks related," and production AI systems, especially multi-agent ones, need an answer to "what is true, for this entity, as of this moment." Sentra is built to answer the second question, using embeddings as one signal inside a larger resolution process rather than as the entire memory layer.

Building AI Memory That Actually Works

A memory system that survives contact with production needs three architectural properties that go beyond what any embedding model provides.

Write-time resolution. Every piece of incoming information, a Slack message, a CRM update, a document revision, should be resolved against existing knowledge the moment it arrives: which entity does this refer to, does it update or contradict something already known, and what is its valid time range. This is expensive to do well, which is exactly why most teams skip it and pay the cost later at query time, repeatedly, forever.

Bi-temporal structure. The system needs to track two independent timelines: when something was true in the world (valid time) and when the system learned about it (transaction time). This is what lets an agent correctly answer "what did we tell this customer in Q2" even after the underlying policy has since changed, without the two facts colliding. Sentra's context graph is bi-temporal by design, which is what makes it possible to reconstruct state as of any point in time rather than only ever seeing the current, possibly wrong, snapshot.

A shared graph, not a shared index. A vector index stores disconnected chunks. A context graph stores relationships: this person works at this company, this ticket references this contract, this contract supersedes that one. Agents querying a graph get resolved, connected context. Agents querying a flat vector index get a pile of textually similar fragments and have to reconstruct the relationships themselves, badly, every single time.

This is the practical difference between a vector store and what Sentra describes as a company brain: an org-wide memory layer that people and agents both read from and write to, where meaning is resolved once and reused everywhere, rather than re-guessed per query. It is also why the performance numbers diverge so sharply in agentic settings. On Terminal-Bench 2.1, a benchmark designed around realistic multi-step agent tasks, Sentra's write-time-resolved context graph approach reaches roughly 88% task success, because the agent isn't spending its reasoning budget disambiguating stale or conflicting retrieved chunks mid-task. It starts from resolved ground truth.

Closing Takeaway

Embedding models are excellent at one thing: measuring semantic distance between pieces of text. That makes them a genuinely useful tool for fuzzy search, deduplication, and clustering. It does not make them memory. Memory requires resolving who is being talked about, when something was true, which version of a fact is authoritative, and how updates propagate, none of which a similarity score can tell you. Teams that keep patching this gap with more rerankers and bigger context windows are paying for it in latency, token cost, and wrong answers. Teams that resolve meaning once, at write time, in a structured, bi-temporal graph, stop paying that tax on every single query. That is the architectural difference between a vector store and an actual memory system, and it is the reason Sentra was built as a context graph first and a retrieval layer second.