Embed in a route
<FigureFromSource sourcePath="content/publications/figures/products/rag-harness.dsl.md" figureId="RAG-03" />
RAG-03
RAG Retest and Evidence Flow
A retrieval failure is closed only after corpus, ranking, policy, or action controls are changed and replayed.
governed lifecycle
RAG retest lifecycle
- 1Reproduce the failureCapture query, retrieved content, ranking, prompt assembly, model output, and action state.
- 2Identify the control boundaryDetermine whether the failure arises in ingest, retrieval, isolation, provenance, prompt assembly, or action policy.
- 3Change the controlUpdate corpus, ranking, metadata, tenant policy, prompt policy, or tool policy.
- 4Replay the scenarioUse the original and adversarial variants under the changed system.
- 5Review the evidenceConfirm the intended boundary and inspect alternative failure paths.
Retest loop returns to Identify the control boundary
What did the retest prove?
- Closed
- Residual
- Failed
- Inconclusive
About this figure
A retrieval failure is not considered resolved until the underlying control has changed and the scenario has been replayed with evidence. This harness frames RAG incidents as a governed lifecycle: reproduce the failure, identify the control boundary, change the relevant control, replay the same query path, and review the resulting evidence before closing the loop. The control may be corpus coverage, ranking behavior, policy enforcement, or an action guardrail, but the closure rule is the same: the fix must be explicit, testable, and observable. This turns ad hoc troubleshooting into a repeatable retest process that prevents false confidence and ties remediation to verified system behavior.
Citation
RAG Retest and Evidence Flow (RAG-03). AI Security LLC Figure Library. https://aisecurity.llc/publication-dsl/figures/RAG-03