1. Plan: AI Policy & Safe Experimentation
1.1
Inadequate AI Policy
1.10
Incomplete Threat Modeling for AI Systems
1.2
Governance Misalignment
1.3
Inadequate Compliance Mapping
1.4
Undefined Risk Tolerance & Categorization
1.5
Unmonitored AI experimentation
1.6
Insecure Experiment Logging & Monitoring
1.7
Overly Permissive Permissions in Experimentation
1.8
Experiment Output Data Leakage
1.9
Unauthorized / Prohibited Component Usage
2. Code/No Code: AI Asset Discovery
2.1
Incomplete Asset Inventory
2.2
Shadow AI Deployment
2.3
Unidentified Third-Party AI Integrations
2.4
Undocumented Data Flows and Lineage
2.5
Lack of Clarity on AI System Purpose and Criticality
2.6
Overlooked Embedded or Inherited AI Functionality
2.7
Discovery of Outdated or Orphaned AI Assets
3. Build: AI Security Posture Management
3.1
Data Poisoning and Integrity Issues
3.10
Unvetted Use of Open-Source and Third-Party AI Components
3.11
Exposed or Hardcoded Credentials in Build Artifacts
3.12
Failure to Specify or Enforce Secure Model Requirements
3.13
Insufficient Understanding of AI System Boundaries
3.14
Exposed AI Access Credentials in Discovered Assets
3.2
Model Backdoor Insertion or Tampering
3.3
Vulnerable AI Frameworks and Libraries
3.4
Insecure System Prompt Design
3.5
Insecure ML & Data Pipeline Jobs
3.6
Intellectual Property (IP) Theft of Models
3.7
Misclassified or Undocumented Sensitive Data Usage
3.8
Insufficient Human Oversight in Model Development
3.9
Insecure Temporary Artifacts or Intermediate Data Storage
4. Test: AI Red Teaming
4.1
Untested Model
4.2
Incomplete Red-Team Coverage
4.3
Lack of Risk Assessment Process
4.4
Missing Documented Evidence of Red Teaming/Risk Assessment
4.5
Outdated Risk Assessment
4.6
Insecure Storage of Red Teaming Artifacts
4.7
Insufficient Multimodal Security Testing
4.8
Limited Foreign Language Red Teaming
4.9
Limited Scope of Evasion Technique Testing
5. Deploy: Runtime Guardrails
5.1
Insecure API Endpoint Configuration
5.10
Insecure Memory & Logging
5.11
Denial-of-Service (Resource Exhaustion)
5.12
Resource Abuse
5.13
Malicious Content Generation
5.14
Autonomous-Agent Misuse
5.15
Insecure Plugin/Tool Integration
5.16
Cross-Domain Prompt Injection (XPIA)
5.17
Policy-Violating Output
5.2
Unauthorized System Prompt Update/Tampering
5.3
Direct Prompt Injection
5.4
System Prompt Leakage
5.5
Context-Window Overwrite/Manipulation
5.6
Sensitive Data Leakage
5.7
Insecure Output Handling
5.8
Adversarial Evasion
5.9
Model Theft / Extraction
6. Operate: Safe Execution Environment (Sandbox)
6.1
Autonomous Code Execution Abuse
6.10
Autonomous Policy/Compliance Violation
6.2
Unrestricted API/Tool Invocation
6.3
Dynamic/On-the-Fly Dependency Injection
6.4
Task Decomposition for Policy Evasion
6.5
Indirect Prompt/Instruction Injection
6.6
Autonomous Resource Provisioning/Abuse
6.7
Cross-Agent/Inter-Agent Abuse
6.8
Agentic System Self-Modification
6.9
Covert Channel Use/Evasion
7. Monitor: AI Activity Tracing
7.1
Insufficient AI Interaction Logging
7.2
Missing Real-time Security Alerts
7.3
Undetected Model Drift/Degradation
7.4
Inadequate AI Audit Trails
7.5
Data Exfiltration via Monitoring/Telemetry
7.6
Absence of AI-Specific Incident Response Plan

SAIL

/

Deploy - Runtime Guardrails

/

Adversarial Evasion

5.8

.

Adversarial Evasion

sail
5.8
Download
Risk

Adversarial Evasion

Description

Attackers craft inputs that evade model or runtime guardrails, causing misclassification or bypassing abuse filters.

Example

Adversary submits obfuscated harmful input that escapes detection and is processed by the model.

Assets Affected

Model Inference endpoint

Model Response

Mitigation
  • Adversarial training
  • Input filtering
  • Continuous testing
  • Update abuse detection mechanisms
Standards Mapping
  • ISO 42001: A.6.2.6, A.9.4
  • NIST AI RMF: MEASURE 2.6, MEASURE 2.7
  • DASF v2: MODEL SERVING 9.2