Multi-Tenant RAG: Stop One Tenant's Documents Leaking Into Another's

A RAG system that answers questions from customer documents has a failure that a single-tenant demo never shows: the answer is fluent, cited, correct, and built from another customer's file. Nobody sees an error. The user sees a helpful reply, and the leak is discovered when someone recognizes a figure that was never theirs.

This is a data breach that happens to look like a good answer, and it is easy to ship by accident. The vector index holds chunks from every tenant, the search finds whatever is nearest, and the only thing between tenant B and tenant A's contract is a filter that someone remembered to apply.

Below are the places these leaks hide, a retrieval design that makes them structurally hard, and a test you can run in CI. The same thinking applies whether you use a dedicated vector database or the vector extension of your main Postgres.

Where cross-tenant leaks actually come from

Most of them are not exotic attacks. They are ordinary engineering slips, and they cluster in a few places.

• The filter is applied after the search. The index returns the top ten nearest chunks across all tenants, and the code then drops the ones that do not belong. At best the user gets fewer results than expected; at worst a code path forgets the drop. The tenant condition belongs inside the query.
• The filter is optional. A search function that takes an optional tenant parameter will, eventually, be called without one by a script, a test or a new endpoint.
• The model supplies the tenant. If the tenant id is an argument the model fills in when it calls a retrieval tool, a prompt injection can change it. Identity comes from the authenticated session, never from model output.
• Caches keyed on the question. A semantic or response cache that stores an answer under the question text will happily serve tenant A's answer to tenant B asking the same thing. Cache keys need the tenant.
• Shared chat history and memory. Conversation stores, summaries and any long-term memory need the same scoping as documents.
• Logs and traces. Retrieved chunks written to a shared observability tool put customer content where support staff and other tenants' debugging sessions can see it.

Make the tenant part of the call signature

The most reliable design is one where it is impossible to express an unscoped search. Wrap the vector store in a single retrieval function whose first parameter is a tenant context object, built once per request from the session. Application code never touches the raw index client.

// the only way to read chunks
async function retrieve(ctx: TenantContext, query: string, k = 8) {
  const vec = await embed(query)
  const rows = await index.search(vec, {
    limit: k,
    where: { tenant_id: ctx.tenantId },   // inside the query
  })
  // belt and braces: refuse to return a foreign chunk
  return rows.filter((r) => r.tenant_id === ctx.tenantId)
}

// the model-facing tool takes only the query
const searchTool = (ctx) => ({
  name: 'search_documents',
  run: ({ query }) => retrieve(ctx, query),
})

Two details do the work. The tenant filter runs inside the search, so the nearest-neighbor ranking only considers that tenant's chunks. And the tool exposed to the model closes over the context, so the model has no parameter through which to name another tenant. The second filter after the search looks redundant, and it is. It exists so that a bug in the first one fails loudly rather than silently, and you should log a security event whenever it removes anything.

Namespaces, metadata filters or separate indexes

There are three ways to separate tenants in the store, and they trade isolation against cost and operational effort.

• Metadata filter on a shared index. Cheapest and simplest to operate. Isolation depends entirely on the filter being correct everywhere, so the wrapper above is not optional. Some approximate-search indexes handle highly selective filters poorly, returning too few results, so test this with your smallest tenant.
• Namespace or partition per tenant. The store itself partitions the data, so a missing filter returns nothing rather than everything. Better failure mode, with more management overhead as tenant count grows.
• Separate index or database per tenant. The strongest isolation, which also makes deletion and export simple. It costs more and is harder to operate in large numbers. The tradeoff mirrors the one in shared schema versus database per tenant, and the same reasoning applies: regulated customers and very large tenants may justify it, long-tail tenants usually do not.

We have run a document pipeline feeding a vector store of roughly 21,000 embedded chunks for in-app retrieval and chat, and operated 700+ tenant databases across our platforms. The lesson we would pass on is not a product choice. It is that isolation should be enforced by structure, with a single choke point, because the discipline of remembering does not survive the fifth developer.

Permissions inside a tenant matter too

Tenant isolation is the outer boundary. Inside a tenant, not everyone may read every document: the finance folder, an HR file, a draft contract. If retrieval ignores document-level permissions, the assistant becomes a way around them, since a user who cannot open a file can ask a question whose answer is in it.

Store the access rule with each chunk (owner, group or role list) and filter on the requesting user's permissions in the same query. Re-evaluate when permissions change, since stale ACLs on indexed chunks are the usual bug. Chunking choices affect this too; if you split documents so that a chunk loses its parent's permissions, you have lost the thing to filter on, which is one reason to keep metadata attached during chunking structured documents.

Check citations before you render them

A last control sits at the output. If the answer cites chunk or document ids, verify that every cited id belongs to the requesting tenant and user before showing it. This catches leaks introduced anywhere upstream, including the ones you have not thought of. Treat retrieved text as untrusted as well, since a document can contain instructions aimed at your model; our post on prompt injection in customer documents covers that side.

A test you can run in CI

Plant canaries. Create two test tenants and give each a document containing a unique, meaningless string, such as a made-up product code. Then run the full stack as tenant B with questions designed to pull tenant A's string: the exact wording of A's document, a paraphrase, and a prompt that explicitly asks for other customers' data. The assertion is plain: A's canary must never appear in any answer, citation, log line or cache entry produced for B.

Run it on every deploy, and run it again after any change to indexing, caching or the retrieval wrapper. It takes minutes to write and it converts a silent, reputation-ending bug class into a failing build.

When this is more than you need

If your assistant only searches public or shared material, such as your own help center, there are no tenants to separate and most of this does not apply. If you have a handful of customers and each gets a fully separate deployment, isolation is provided by the infrastructure. The structure above is for the common middle: one shared system, many customers, each with private files. If you are designing that kind of product, our work on AI and agentic systems includes the retrieval layer and its tests, and the way we scope it is on our approach page.