AI incidents are harder to diagnose than ordinary service outages because the application can stay online while behavior becomes wrong, unsafe or unexpectedly expensive.

Classify the incident first

Separate provider outage, latency regression, retrieval contamination, tool failure, policy regression, model behavior change and cost anomaly. Contain before optimizing. Disable high-risk tools, route to a known fallback model, reduce retrieval scope or temporarily degrade to text-only behavior when necessary.

Preserve evidence

Keep trace IDs, model versions, policy versions and sanitized tool-call metadata so the team can reconstruct the failure. Use feature-level kill switches.

Teams should be able to disable one agent action or model route without taking the entire product offline. Multi-region fallback patterns are covered in Multi-Region AI Failover. Review customer impact. Count affected tasks and consequential side effects, not just HTTP errors.

Design retrieval around provenance

Every chunk should carry enough metadata to reconstruct its origin: source document, version, section, ingestion time and any transformation applied before embedding. Provenance makes deletion, correction and debugging far more reliable than treating the vector store as an opaque collection of text fragments. It also enables source-aware ranking. Current first-party documentation can outrank an old mirrored copy even when both are semantically similar. Use multiple stages instead of one similarity score. Embedding similarity is a useful candidate generator, not a complete evidence policy. Keyword signals, metadata filters, reranking and freshness checks can improve the final set of context provided to the model. The last stage should optimize for useful evidence diversity rather than simply the highest individual scores. This reduces repeated context and improves coverage of multi-part questions.

Measure failure at the right layer. A final answer can be wrong even when retrieval was correct, and retrieval can be poor even when the model manages to guess the right answer. Evaluate corpus coverage, retrieval recall, reranker quality, citation support and answer correctness separately. This decomposition makes optimization more efficient because teams can change the layer that actually caused the failure instead of tuning prompts for indexing problems. Operate updates and deletion as first-class workflows. Document lifecycle events should trigger deterministic propagation through embeddings, caches and derived summaries. Monitoring should detect orphaned chunks or stale versions so removed information does not remain retrievable accidentally.

Treat retrieval as a data lifecycle, not a single query

Reliable RAG begins before the user asks a question. Documents are ingested, parsed, chunked, enriched, embedded, indexed, updated and eventually deleted. Every stage can create stale or duplicate state, so the system needs version identifiers and lineage that connect a retrieved chunk back to the source document and ingestion run. This lineage makes corrections practical. When a document changes, the platform can identify which chunks, embeddings and caches depend on the old version instead of rebuilding or trusting the entire index blindly. Separate candidate retrieval from evidence selection. High recall is useful in the first retrieval stage, but the final prompt should contain only evidence that is relevant, current and sufficiently distinct. Reranking, deduplication and source-quality rules can prevent repeated or low-value chunks from crowding out better evidence. The system should also know when evidence is insufficient. Returning fewer high-quality chunks and asking for clarification is often safer than filling the context window with loosely related text.

Control redundancy before it reaches the model. Mirrored pages and repeated chunks can dominate similarity scores and make duplicated wording appear to be stronger evidence. Diversity-aware reranking and per-source limits preserve context space for distinct facts.

More retrieved tokens are not automatically better; compact evidence often produces clearer grounding at lower cost.

Regression-test the retrieval pipeline

Re-chunking, re-embedding and large ingestion changes can shift results even when the model is unchanged. Run representative queries after corpus changes and compare which sources and passages are retrieved. This turns data maintenance into an observable release process instead of an invisible source of quality drift.

Preserve provenance through the entire data path. A retrieved chunk should remain tied to its document, section, version and ingestion run. Provenance allows teams to explain stale answers and to retire the embeddings, caches and summaries that depend on an outdated source.

This also enables source-aware ranking so current first-party material can outrank older mirrors or duplicated copies.

Separate broad retrieval from final evidence

Similarity search is useful for candidate recall, but the final prompt should be compact. Rerank for relevance, authority, freshness and diversity, and remove repeated passages before generation. Evaluation should measure retrieval recall, source freshness and answer groundedness separately so indexing problems do not get mistaken for model problems.

Bottom line. An AI incident runbook should connect behavioral quality, infrastructure, retrieval and tools so teams can contain failures quickly and learn from them.