EVIDENCE
AI Security Eval Coverage Evidence
Prove your AI security tests cover the risks that matter.
Measure whether your AI evaluations actually cover prompt injection, indirect prompt injection, tool abuse, RAG poisoning, data leakage, tenant crossover, memory poisoning, guardrail bypass, and auditability. Eval Coverage Auditor audits coverage across existing eval systems; it does not replace promptfoo, garak, deepeval, Inspect, or internal eval runners.
Measure
Identify which AI security domains are covered by existing evals.
Expose
Find missing tests for prompt injection, tool abuse, RAG, memory, tenant isolation, and unsafe outputs.
Prove
Generate coverage summaries that support security review, buyer diligence, and release readiness.
Connect
Turn Prompt Asset, RAG, Tool Capsule, and Threat Canvas findings into eval requirements.
Core capabilities
What Eval Coverage Auditor does.
Coverage Matrix
Read evaluation files, test names, rubrics, fixtures, and scenarios to map coverage to AI security risk domains.
Missing Domain Findings
Identify absent coverage for direct and indirect prompt injection, tool abuse, RAG poisoning, data leakage, tenant crossover, memory poisoning, guardrail bypass, unsafe outputs, and auditability.
Release Readiness Signals
Highlight missing release-blocking tests and regression gaps for model, provider, prompt, retrieval, and tool changes.
Finding-to-Eval Requirements
Convert Threat Canvas, Tool Capsule, Prompt Asset, RAG, and Permission findings into recommended eval backlog items.
Evidence Summary
Produce buyer-ready summaries that show what has been tested, what has not, and what must be added before security review or release.
Eval Runner Neutral
Measure coverage across existing eval runners and internal test systems without forcing a new execution engine.
Evidence & signals
What you get out of the box.
Risk Domains
- Prompt injection
- Indirect prompt injection
- Tool abuse
- RAG poisoning
- Data leakage
- Tenant crossover
- Memory poisoning
- Guardrail bypass
- Auditability
Inputs
- Eval files
- Test names
- Rubrics
- Fixtures
- Scenarios
- Internal eval metadata
Deliverables
- Eval coverage matrix
- Missing domain findings
- Release readiness signals
- Recommended eval backlog
- Evidence-ready summary
AI SECURITY WORKBENCH
Ready to put Eval Coverage Auditor to work?
Eval Coverage Auditor is an active-development Workbench capability available through scoped public-site review conversations. It audits coverage across your existing eval systems and turns missing risk domains into evidence-ready backlog.
Continue through the Workbench
Previous MADE stage
Runtime Trace
Capture and reconstruction of prompts, retrieval, model calls, identities, approvals, tools, outputs, and side effects.
Related experience
AI Security Program Scorecard
14-domain AI product security baseline with evidence pack generation.
OEM & partner
OEM Engine
Embed selected Workbench capabilities through bounded APIs, SDKs, and lifecycle contracts.