Labs
AI Security Range
LLM Attack Range
Scenario execution, attack-pack coverage, and control-evidence readiness in one dedicated product surface.
Curated from the upstream local-first lab package and imported into this repo as a public product shell.
Scenario files
157
Active attack packs
15
Tool adapters
8
Content batches
7
Public boundary
Public-safe counts, labels, snapshots, and chart exports only. Executable runs, private evidence, and orchestration stay upstream.
Registry snapshot generated 2026-05-20T11:15:24.948Z.
Overview
The range is a curated product snapshot, not a loose report export.
Unique attack scenarios run at least once in this seeded window.
Scenarios focused on synthetic media, output integrity, and multimodal abuse.
Share of seeded attempts that reached defined exploit objective pre-mitigation.
Runs with prompt, tool-call, decision-log, and remediation metadata present.
multimodal_exfiltration
097-image-steganography-exfil
Severity critical · deepfakes_synthetic_media
Success 55% · Controls 44% · Evidence 81%
prompt_injection
001-prompt-injection-basic
Severity high · prompt_and_generation_security
Success 42% · Controls 68% · Evidence 88%
synthetic_media_abuse
154-deepfake-script-generation
Severity high · deepfakes_synthetic_media
Success 39% · Controls 52% · Evidence 86%
multimodal_jailbreak
100-video-frame-injection
Severity high · prompt_and_generation_security
Success 34% · Controls 61% · Evidence 84%
output_integrity
090-technical-doc-falsification
Severity high · data_privacy_and_provenance
Success 31% · Controls 63% · Evidence 85%
output_integrity
086-citation-fabrication
Severity medium · data_privacy_and_provenance
Success 28% · Controls 74% · Evidence 90%
Registry snapshot
Public signal stack
157 scenarios carry explicit ISO 42001, EU AI Act, NIST AI RMF, and MITRE ATLAS coverage.
157
scenario files
Attack packs
15 active namespaces surface distinct attack families.
100% signal
Tool adapters
All eight adapters are first-class and producing findings.
100% signal
Content batches
Batches 01 through 07 are represented in the public snapshot.
100% signal
Coverage
Control coverage is explicit across ISO 42001, EU AI Act, NIST AI RMF, and MITRE ATLAS.
100% signal
Boundary note
Lab outputs are directional scenario evidence and not proof of any individual company's internal security maturity.
Snapshot details
Status
mock_seeded
As of
2026-05-20
Run window
2026-01-01 → 2026-04-30
Registry generated
2026-05-20T11:15:24.948Z
Framework coverage
Scenario families
The scenario corpus is broad enough to support real validation paths.
Family
Prompt injection
Family
Data exfiltration
Family
Excessive agency
Family
RAG poisoning
Family
Alignment bypass
Family
Supply chain
Family
Output handling
Family
Multimodal
Family
Governance bypass
Attack packs
Fifteen active attack-pack namespaces now surface as public taxonomy instead of hidden depth.
Taxonomy
Alignment
Benchmark gaming, rule contradiction, dark-pattern consent, and evaluator manipulation.
Taxonomy
Model integrity
Backdoor, inversion, membership, drift, and extraction probes.
Taxonomy
Prompt injection
Direct overrides, multi-turn escalation, bidi evasion, and agent-collusion probes.
Taxonomy
DoS
Context floods, token exhaustion, recursive callbacks, and tool amplification.
Taxonomy
Indirect prompt injection
Hidden instructions inside retrieved or forwarded content.
Taxonomy
RAG poisoning
Poisoned retrieval content, source spoofing, and boundary-crossing attempts.
Taxonomy
Data exfiltration
Secret harvesting, tool-output scraping, and context overreach.
Taxonomy
Cross-tenant leakage
Tenant alias confusion and scope escalation attempts.
Taxonomy
Delegated authority
Authority laundering between planner and executor agents.
Taxonomy
Memory poisoning
Persistent trust-anchor corruption in long-lived memory stores.
Taxonomy
Browser agent
DOM comments, fake reauth loops, and origin-confusion coercion.
Taxonomy
Tool abuse
Unsafe tool-call prompting and action-chain abuse.
Taxonomy
Agent sandbox escape
REPL breakout, path traversal, and SSRF via tool usage.
Taxonomy
Supply chain poisoning
MCP configs, malicious PR instructions, and typosquatted dependencies.
Taxonomy
Multimodal injection
OCR, EXIF, and steganographic payloads.
Pack summary
Active packs
15
Threat vectors
22
Published docs
5
Raw payloads
0
The public shell surfaces labels, counts, and family signals. Raw payload content stays upstream so the site remains safe to publish.
Tool coverage
All eight adapters are first-class and fully wired.
Taxonomy
promptfoo
Assertion-driven conversational evaluation and regression coverage.
Taxonomy
garak
Scanner coverage for leakage, jailbreak, and behavior probes.
Taxonomy
PyRIT
Objective-driven red-team orchestration and multi-turn attack flows.
Taxonomy
AgentDojo
Agent trajectory logging, tool use, and permission-boundary exercises.
Taxonomy
Giskard
Targeted eval surface for benchmark-style checks.
Taxonomy
Inspect AI
Targeted evaluation and scenario review.
Taxonomy
NeMo Guardrails
Policy-pack and refusal behavior validation.
Taxonomy
OpenAI Evals
Staged benchmark harness for future adapter parity.
Tool matrix
First-class tools
8
Total adapters
8
All eight adapters — promptfoo, garak, PyRIT, AgentDojo, Giskard, Inspect AI, NeMo Guardrails, and OpenAI Evals — are first-class and producing findings.
Content libraries
The upstream docs now break the range into clear batch-era content libraries.
Taxonomy
Batch 01 - Initial Scenarios and Assertions
Seed corpus, assertion scaffolding, and the initial scenario set.
Taxonomy
Batch 02 - Expanded Attack Packs and Scenarios
Adds indirect injection, exfiltration, cross-tenant leakage, memory poisoning, browser agent, and delegation coverage.
Taxonomy
Batch 03 - Agentic and Multimodal
Adds multimodal injection, supply chain poisoning, agent sandbox escape, recursive DoS, and model inversion.
Taxonomy
Batch 04 - Hardening and Framework Alignment
Adds remediation blocks, hardened variants, industry-specific content, honeytokens, and framework alignment.
Taxonomy
Batch 05 - Operationalization and Governance
Roadmap slot for coverage gates, inferred debt reduction, and CI enforcement.
Taxonomy
Batch 06 - High-Fidelity Scenarios and Metrics
Adds telemetry, latency tracking, stealth and reliability scores, and remediation diffs.
Taxonomy
Batch 07 - Interoperability and Forensics
Adds ECS and SARIF outputs plus portable JSON for SIEM and forensics workflows.
Library summary
Batches
7
Published docs
5
Batch 01 seeds the corpus, batches 02 through 04 expand and harden it, batch 05 stays indexed as a governance roadmap slot, and batches 06 through 07 add telemetry and forensics.
Evidence and replay
The public bundle ships snapshots, charts, and a clear artifact boundary.
Ready
Catalog snapshot
Versioned product catalog for the public labs shell.
Size: 1 JSON file
Ready
Scenario snapshot
Curated scenario rows and control-evidence rollups.
Size: 157 scenario rows
Ready
Overview snapshot
Status, scope, caveat, run window, and data endpoints.
Size: versioned overview
Ready
Metric bundle
Public-safe scorecards and range metrics.
Size: 4 metric cards
Ready
Generation / media rollup
Monthly synthetic-media and prompt-abuse tracking.
Size: 8 monthly rows
Latest generation window
2026-04-01
Monthly synthetic-media and prompt-abuse rollups are included as public-safe snapshot data. The interoperability batch adds ECS and SARIF outputs for downstream forensics.
Controls coverage
Control coverage is tracked alongside the attack surface, not after the fact.
Why this matters
The range is designed to answer a product question: which controls exist, which ones block, and where does evidence still go missing?
- • Explicit coverage across 157 scenarios
- • Zero uncovered controls and zero uncovered ATLAS techniques
- • Weak-evidence controls remain limited to a short, known list
Boundary note
Public counts are directional. They support product exploration and training, not claims about real-world customer maturity.
Weak evidence controls
Gap register
Uncovered controls: 0. Uncovered ATLAS techniques: 0.
Provenance
The product shell is public-safe. The runtime stays upstream.
AI Security Range
Curated from the upstream local-first lab package and imported into this repo as a public product shell.
Public-safe counts, labels, snapshots, and chart exports only. Executable runs, private evidence, and orchestration remain upstream.
This is a curated snapshot, not a claim that the executable runtime is hosted in this repo.
Public outputs
- • Scenario snapshot and overview JSON
- • Public metrics and generation rollups
- • Chart exports for the lab signal surface
- • Curated product catalog, attack packs, and library index
v1 · Public-safe snapshot only.
Private engagement
Turn scenarios into a red-team sprint.
Use controlled attack scenarios to scope a private test plan for prompt injection, RAG abuse, agent misuse, unsafe actions, and policy bypass. Findings include evidence, severity notes, and mitigation guidance.