SECENG WORKBENCH

Retrieval, Context & XPIA Testing

Test whether your RAG system retrieves what it should — and nothing it should not.

SecEng RAG Test Harness maps retrieval boundaries and validates authorization, provenance, context integrity, tenant isolation, poisoned content, source laundering, and indirect prompt injection across RAG-backed AI systems.

Test RAG Boundaries Scope RAG Review

WAS RETRIEVAL AUTHORIZED?

Boundary Testing

Cross-tenant, cross-role, stale-permission, and revoked-user retrieval cases.

Source Provenance

Validate citations, source lineage, and source-laundering paths.

Privacy Gates

Check whether sensitive or regulated chunks enter retrieved context.

Poison Testing

Test poisoned documents and indirect prompt injection before context reaches the model.

SecEng Workbench · Retrieval & Context Security

SecEng RAG Test Harness

Tenant-boundary tested

Boundary Testing

Cross-tenant, cross-role, stale-permission, and revoked-user retrieval cases.

Source Provenance

Validate citations, source lineage, and source-laundering paths.

Privacy Gates

Check whether sensitive or regulated chunks enter retrieved context.

Poison Testing

Test poisoned documents and indirect prompt injection before context reaches the model.

Leakage categories tracked

Capture query → retrieval → policy check → context window → response, and classify leakage events by type before they reach the model.

Cross-tenantCross-roleStale permissionPoisoned contentSource laundering

SecEng Workbench · Retrieval & Context Security

SecEng RAG Test Harness

Tenant-boundary tested

Boundary Testing

Cross-tenant, cross-role, stale-permission, and revoked-user retrieval cases.

Source Provenance

Validate citations, source lineage, and source-laundering paths.

Privacy Gates

Check whether sensitive or regulated chunks enter retrieved context.

Poison Testing

Test poisoned documents and indirect prompt injection before context reaches the model.

Leakage categories tracked

Capture query → retrieval → policy check → context window → response, and classify leakage events by type before they reach the model.

Cross-tenantCross-roleStale permissionPoisoned contentSource laundering

Core capabilities

What SecEng RAG Test Harness does.

Identity cases

Test cross-user, cross-tenant, revoked user, contractor, admin, and external guest access cases. Validate that retrieval respects identity and authorization — not just query relevance scores.

Tenant-boundary tests

Seed fixture documents scoped by tenant, role, and permission state. Confirm retrieval gates enforce boundaries and do not bleed across tenants on revoked or stale permissions.

Poisoned-corpus checks

Seed poisoned fixture documents and indirect prompt-injection content. Confirm hostile content is blocked before it enters the context window and influences model responses.

Context-leak checks

Capture query → retrieval → policy check → context window → response. Classify leakage events by type: cross-tenant, cross-role, stale permission, poisoned content, source laundering.

Full Pipeline Evidence Capture

Produce structured evidence showing what entered context, what was blocked, and what leaked. Export as boundary test plan, identity test cases, and evidence pack.

RAG-Specific Regression Harness

Turn test results into replayable regression fixtures. Build a permanent authorization test suite that runs on every corpus or model update.

Evidence & signals

What you get out of the box.

What it produces

Boundary test plan
Identity and tenant test cases
Poisoned corpus scenarios
Source provenance findings
Context leakage findings
Evidence pack export
Regression harness bundle

Leakage categories

Cross-tenant
Cross-role
Stale permission
Poisoned content
Source laundering
Indirect prompt injection

Export formats

Evidence Pack (ZIP)
Results Report (PDF)
Control Mapping (CSV)
Query / Chunk Log (JSON)

Red team + Blue team

Built for both sides of the security equation.

Red Team Use

Demonstrate cross-tenant retrieval using fixture documents scoped to restricted tenants
Show stale-permission access: fixture files retrieved after authorization was revoked
Inject poisoned fixture documents and confirm whether they influence model responses

Blue Team Use

Export ACL evidence, source provenance reports, and policy-check audit logs
Build RAG regression suites that run automatically on every corpus or model update
Map retrieval findings to governance controls for buyer-ready evidence

Retrieval creates a chain of trust boundaries.

RAG security depends on more than the model response. Risk can enter through content ingestion, document identity, metadata, storage, indexing, retrieval, ranking, context assembly, prompt construction, generation, and downstream actions.

RAG-01imported graph

Retrieval Trust Topology

RAG security depends on the full path from query and corpus boundaries through ranking, prompt assembly, model behavior, and output.

Mapping these boundaries reveals where authorization, provenance, tenant isolation, filtering, citation, and action controls must hold for the final answer to be trustworthy.

Test the system by failure class, not by jailbreak count.

A complete RAG review examines how untrusted content, identity mistakes, retrieval manipulation, metadata abuse, cross-tenant access, context leakage, unsafe generation, and downstream actions can combine.

RAG-02coverage matrix

RAG Failure-Mode Coverage

A useful harness separates prompt injection, corpus poisoning, cross-tenant retrieval, provenance loss, and unsafe action propagation.

Failure modes

Ingest

Retrieve

Assemble

Generate

Act

Retest

Indirect prompt injection

Hostile instructions enter through retrieved content.

Covered

Partial

Covered

Corpus poisoning

Malicious or misleading content affects retrieval and generation.

Covered

Partial

Covered

Partial

Covered

Cross-tenant retrieval

Content crosses an intended tenant or access boundary.

Partial

Covered

Partial

Planned

Provenance loss

The system cannot preserve or communicate source origin and trust.

Covered

Partial

Unknown

Covered

Unsafe action propagation

Retrieved content influences a tool or consequential action.

Not applicable

Partial

Covered

Partial

Indirect prompt injection

Hostile instructions enter through retrieved content.

Ingest: Covered
Retrieve: Covered
Assemble: Covered
Generate: Covered
Act: Partial
Retest: Covered

Corpus poisoning

Malicious or misleading content affects retrieval and generation.

Ingest: Covered
Retrieve: Covered
Assemble: Partial
Generate: Covered
Act: Partial
Retest: Covered

Cross-tenant retrieval

Content crosses an intended tenant or access boundary.

Ingest: Partial
Retrieve: Covered
Assemble: Covered
Generate: Covered
Act: Partial
Retest: Planned

Provenance loss

The system cannot preserve or communicate source origin and trust.

Ingest: Covered
Retrieve: Covered
Assemble: Covered
Generate: Partial
Act: Unknown
Retest: Covered

Unsafe action propagation

Retrieved content influences a tool or consequential action.

Ingest: Not applicable
Retrieve: Partial
Assemble: Covered
Generate: Covered
Act: Covered
Retest: Partial

Coverage states

CoveredPartialPlannedUnknownNot applicable

The matrix must be reviewed against the live harness. A named failure mode is not evidence that every pipeline stage is covered.

The goal is not to maximize prompt variants. It is to determine which failure classes are reachable in this architecture, what controls should prevent them, and what evidence supports the conclusion.

Live fixture route

Walk through a retrieval security test run with a live ACME Corp fixture.

Open the demo to explore tenant boundary tests, leakage events, corpus inventory, and framework coverage — all fixture-driven.

Test RAG Boundaries Scope RAG Review

A reproduced RAG failure should become durable evidence.

Preserve the source document or corpus condition, retrieval trace, assembled context, model behavior, control state, consequence, remediation, and retest result as one connected case.

RAG-03

RAG Retest and Evidence Flow

A retrieval failure is closed only after corpus, ranking, policy, or action controls are changed and replayed.

RAG retest lifecycle

1
Reproduce the failure
Capture query, retrieved content, ranking, prompt assembly, model output, and action state.
2
Identify the control boundary
Determine whether the failure arises in ingest, retrieval, isolation, provenance, prompt assembly, or action policy.
3
Change the control
Update corpus, ranking, metadata, tenant policy, prompt policy, or tool policy.
4
Replay the scenario
Use the original and adversarial variants under the changed system.
5
Review the evidence
Confirm the intended boundary and inspect alternative failure paths.

Retest loop returns to Identify the control boundary

What did the retest prove?

Closed
Residual
Failed
Inconclusive

This turns a one-time demonstration into an engineering artifact that can guide remediation, verify isolation and provenance controls, prevent regression, and support a release or risk decision.

Related Workbench tools

Code Scanner Adversarial Range Runtime Proxy Evidence Packs Launch Security Review

SECENG WORKBENCH

Test retrieval boundaries before RAG failures reach production.

Scope a RAG review or guided test harness build for tenant isolation, provenance, poisoned content, and context-leakage risks.

Start No-Cost Scoping See the full Workbench