David Wolf · Project Use Case
AI SECURITY · PRODUCT SECURITY · CONFIDENTIAL AI GOVERNANCE PROGRAM
Confidential AI Governance Program
AI Governance Controls with Garak, NeMo Guardrails, Presidio & Promptfoo
Implementing practical AI control evidence for ISO 42001, NIST AI RMF, AIMS, agent identities, permissions, red teaming, privacy, and output evaluation.
Designed a practical AI governance control layer using Garak, NeMo Guardrails, Microsoft Presidio, Promptfoo, agentic identities, permission scoping, evaluation gates, and evidence-generation workflows to support ISO 42001, NIST...
Client
Confidential / Internal AI Governance Program
Engagement Type
Consulting / Research / Buildout
Period
2025–2026
Role
AI Product Security Architect / AI Governance Engineer
Focus Areas
AI Governance Engineering, ISO 42001 Control Implementation, NIST AI RMF Control Implementation, AIMS-Style AI Management Systems
The Research Narrative
Strategic Problem
Frameworks such as ISO 42001, NIST AI RMF, and AIMS-style AI management systems describe what responsible management should achieve, but they do not automatically produce engineering controls. The challenge...
What David Did
David designed a practical control stack: Garak-style adversarial tests for unsafe model behavior, NeMo Guardrails-style policy flows for conversation and action constraints,...
What Became Clearer
The result was a practical AI governance engineering pattern. Instead of slowing delivery with abstract compliance language, the system created repeatable tests, evidence artifacts,...
Consulting Proof
This is evidence of turning messy security telemetry into explainable dashboards, alert-quality improvements, and executive-ready operating views.
The Context
AI governance cannot remain a binder of policies once systems become agentic. When models can retrieve data, call tools, trigger workflows, and act across systems, governance needs enforceable controls: red-team tests, guardrails, privacy checks, permission boundaries, logs, scoring, and acceptance evidence.
The Challenge
Frameworks such as ISO 42001, NIST AI RMF, and AIMS-style AI management systems describe what responsible management should achieve, but they do not automatically produce engineering controls. The challenge was converting governance language into technical control points that could be tested and evidenced.
What I Did
David designed a practical control stack: Garak-style adversarial tests for unsafe model behavior, NeMo Guardrails-style policy flows for conversation and action constraints, Presidio-style PII handling for sensitive data protection, Promptfoo-style evaluation suites for regression testing, and agent identity and permission models for tool access control.
- •Mapped ISO 42001, NIST AI RMF, and AIMS-style governance objectives into concrete product-security and AI-security control categories
- •Defined a practical AI control architecture covering model evaluation, red-team testing, prompt-injection testing, guardrails, PII handling, agent identity, tool permissions, logging, and acceptance gates
- •Used Garak-style adversarial testing patterns to probe model and application behavior for unsafe, unreliable, or policy-violating responses
- •Used NeMo Guardrails-style conversational guardrails to define allowed flows, blocked flows, topic constraints, refusal behavior, and policy routing
- •Used Microsoft Presidio-style PII detection and redaction patterns to identify sensitive data before prompts, tool calls, logs, or downstream storage
- •Used Promptfoo-style evaluation suites to make prompt, model, workflow, and regression tests repeatable
- •Defined agentic identity patterns separating human user identity, service identity, agent identity, workflow identity, and delegated tool permissions
- •Specified least-privilege permission boundaries for agent tools, connectors, files, APIs, browser automation, and workflow execution
The Outcome
The result was a practical AI governance engineering pattern. Instead of slowing delivery with abstract compliance language, the system created repeatable tests, evidence artifacts, scoring, and review gates. That made AI governance more useful to product teams, security teams, risk owners, and executives.
Research Outcomes
Signal Quality
Improved the trustworthiness of operational security signals
IAM Clarity
Normalized identity telemetry and debugged alert behavior
Stakeholder Visibility
Made technical risk and status easier to explain
Operational Impact
Turned raw telemetry into actionable security intelligence
Capabilities Demonstrated
IAM / Access Control
Identity telemetry and access insights
Operational Reporting
Actionable views for security operations
Public-Safe Evidence
Shareable insights without sensitive data
Security Analytics
Signal investigation and event analysis
SIEM Alert Debugging
Noise reduction and signal validation
Dashboard Development
Operational and executive views
Executive Reporting
Security data translated for leadership
Telemetry Normalization
Consistent and trusted data
Key Deliverables
- •AI governance control architecture
- •ISO 42001 to technical control mapping
- •NIST AI RMF to technical control mapping
- •AIMS-style AI management system control mapping
- •Garak-style adversarial testing plan
- •NeMo Guardrails-style policy and conversational control patterns
- •Presidio-style PII detection and redaction workflow
- •Promptfoo-style evaluation suite model
Tools & Technologies
Consulting Translation
The reusable pattern is not Disney-specific: normalize fragmented security telemetry, debug low-signal alert behavior, build trusted operating views, and give leadership evidence they can act on without exposing sensitive systems.