Retrieval-augmented generation can fail even when the model, vector database and application are technically healthy. One of the quietest failure modes is stale knowledge: the system retrieves a document that was correct last month but is wrong today. Because the answer may still sound plausible, stale retrieval can be harder to detect than an obvious outage. Production RAG systems should therefore treat freshness as a data-quality and reliability problem, not as a background indexing detail.

Track more than the document modification time

A useful freshness model records several timestamps: when the source changed, when ingestion detected the change, when the document was transformed, when embeddings were regenerated and when the new version became searchable. These timestamps reveal where delay actually occurs. If the source changed at 09:00 but the vector index was not updated until 15:00, the system had a six-hour stale window even though the pipeline eventually succeeded.

Assign freshness classes by source. Not every document needs the same service level. Pricing, inventory, policy, account status and operational runbooks may require near-real-time or hourly refresh. Long-lived technical documentation may be acceptable with daily or weekly indexing. Classifying sources prevents teams from wasting resources on constant re-indexing while still protecting high-change information.

Store version identity with each chunk

Chunking often hides document versioning. If an updated source is indexed without removing or superseding older chunks, retrieval may return both versions. Each chunk should therefore carry a source identifier, version or content hash, update time and active/inactive status. This also makes rollback safer when a source update turns out to be incorrect.

Detect stale knowledge before users do. Create a small set of freshness tests around facts that change regularly. Examples include current plan limits, product versions, policy dates or pricing values. Run these tests after ingestion and on a schedule. If the system still answers with an old value, inspect retrieval ranking, caching, duplicate chunks and source synchronization rather than assuming the language model is hallucinating.

Monitor ingestion lag as an SLO

Freshness should have measurable objectives. A team might define that 99% of priority documents become searchable within 10 minutes of source change, while low-priority documentation can tolerate six hours. Useful metrics include source-to-index lag, percentage of documents beyond SLA, failed ingestion count, retry delay and number of superseded chunks still retrievable.

Do not ignore caches. A vector index can be fresh while the final answer remains stale because another layer caches retrieval results, prompt context or completed responses. Teams should map all caching layers and define invalidation behavior for high-change facts. Model context caching can reduce cost, but only when teams understand which context is safe to reuse. Our article on model context caching discusses that tradeoff in detail.

Permissions and freshness interact

When a document changes permissions, stale indexing becomes a security issue rather than merely an accuracy problem. A user who lost access to a source should not continue retrieving old chunks from the vector store. That is why freshness pipelines should propagate both content updates and access-control changes. See our guide to RAG permission filtering for the isolation side of the problem.

Use incremental indexing carefully. Incremental updates are cheaper than rebuilding the entire index, but they require strong change detection. Teams need to know whether a file was added, modified, moved, deleted or simply renamed. Deletion is especially important. A source that disappears should eventually disappear from retrieval too, unless retention policy explicitly says otherwise.

A practical operating checklist

• Define freshness classes for each data source.
• Record source, ingestion and index timestamps.
• Version chunks and retire superseded content.
• Run regression tests on recently changed facts.
• Monitor source-to-index lag and stale-document rate.
• Invalidate caches when high-priority facts change.
• Propagate access-control updates with content updates.

Add ownership for stale-source incidents

Freshness problems often fall between teams because the source application, ingestion service and retrieval layer may have different owners. Assign a clear owner for each source and define who responds when freshness SLOs are missed. An incident should identify whether the source failed to publish, the connector failed to ingest, indexing stalled or a cache served an older result. Ownership turns freshness from an informal data-engineering concern into an operational responsibility. It also makes post-incident reviews more useful because teams can fix the exact stage that created the stale window.

Freshness is part of answer quality. A RAG system is not reliable simply because it retrieves relevant text. The text must also be current enough for the decision being made. Once teams measure freshness explicitly, stale answers stop being mysterious model failures and become an observable operational problem that can be managed with normal reliability practices.