1. Plan: AI Policy & Safe Experimentation
1.1
Inadequate AI Policy
1.10
Incomplete Threat Modeling for AI Systems
1.2
Governance Misalignment
1.3
Inadequate Compliance Mapping
1.4
Undefined Risk Tolerance & Categorization
1.5
Unmonitored AI experimentation
1.6
Insecure Experiment Logging & Monitoring
1.7
Overly Permissive Permissions in Experimentation
1.8
Experiment Output Data Leakage
1.9
Unauthorized / Prohibited Component Usage
2. Code/No Code: AI Asset Discovery
2.1
Incomplete Asset Inventory
2.2
Shadow AI Deployment
2.3
Unidentified Third-Party AI Integrations
2.4
Undocumented Data Flows and Lineage
2.5
Lack of Clarity on AI System Purpose and Criticality
2.6
Overlooked Embedded or Inherited AI Functionality
2.7
Discovery of Outdated or Orphaned AI Assets
3. Build: AI Security Posture Management
3.1
Data Poisoning and Integrity Issues
3.10
Unvetted Use of Open-Source and Third-Party AI Components
3.11
Exposed or Hardcoded Credentials in Build Artifacts
3.12
Failure to Specify or Enforce Secure Model Requirements
3.13
Insufficient Understanding of AI System Boundaries
3.14
Exposed AI Access Credentials in Discovered Assets
3.2
Model Backdoor Insertion or Tampering
3.3
Vulnerable AI Frameworks and Libraries
3.4
Insecure System Prompt Design
3.5
Insecure ML & Data Pipeline Jobs
3.6
Intellectual Property (IP) Theft of Models
3.7
Misclassified or Undocumented Sensitive Data Usage
3.8
Insufficient Human Oversight in Model Development
3.9
Insecure Temporary Artifacts or Intermediate Data Storage
4. Test: AI Red Teaming
4.1
Untested Model
4.2
Incomplete Red-Team Coverage
4.3
Lack of Risk Assessment Process
4.4
Missing Documented Evidence of Red Teaming/Risk Assessment
4.5
Outdated Risk Assessment
4.6
Insecure Storage of Red Teaming Artifacts
4.7
Insufficient Multimodal Security Testing
4.8
Limited Foreign Language Red Teaming
4.9
Limited Scope of Evasion Technique Testing
5. Deploy: Runtime Guardrails
5.1
Insecure API Endpoint Configuration
5.10
Insecure Memory & Logging
5.11
Denial-of-Service (Resource Exhaustion)
5.12
Resource Abuse
5.13
Malicious Content Generation
5.14
Autonomous-Agent Misuse
5.15
Insecure Plugin/Tool Integration
5.16
Cross-Domain Prompt Injection (XPIA)
5.17
Policy-Violating Output
5.2
Unauthorized System Prompt Update/Tampering
5.3
Direct Prompt Injection
5.4
System Prompt Leakage
5.5
Context-Window Overwrite/Manipulation
5.6
Sensitive Data Leakage
5.7
Insecure Output Handling
5.8
Adversarial Evasion
5.9
Model Theft / Extraction
6. Operate: Safe Execution Environment (Sandbox)
6.1
Autonomous Code Execution Abuse
6.10
Autonomous Policy/Compliance Violation
6.2
Unrestricted API/Tool Invocation
6.3
Dynamic/On-the-Fly Dependency Injection
6.4
Task Decomposition for Policy Evasion
6.5
Indirect Prompt/Instruction Injection
6.6
Autonomous Resource Provisioning/Abuse
6.7
Cross-Agent/Inter-Agent Abuse
6.8
Agentic System Self-Modification
6.9
Covert Channel Use/Evasion
7. Monitor: AI Activity Tracing
7.1
Insufficient AI Interaction Logging
7.2
Missing Real-time Security Alerts
7.3
Undetected Model Drift/Degradation
7.4
Inadequate AI Audit Trails
7.5
Data Exfiltration via Monitoring/Telemetry
7.6
Absence of AI-Specific Incident Response Plan

SAIL

/

Deploy - Runtime Guardrails

/

Malicious Content Generation

5.13

.

Malicious Content Generation

sail
5.13
Download
Risk

Malicious Content Generation

Description

Model generates harmful, offensive, policy-violating, or illegal content due to insufficient runtime filtering or prompt design.

Example

Model generates hate speech or copyrighted material in response to user queries.

Assets Affected

Model Response

Model Inference endpoint

Mitigation
  • Output filtering
  • Human-in-the-loop review for high-risk queries
  • Content moderation
  • Update prompt/guardrails
Standards Mapping
  • ISO 42001: A.8.2, A.5.4
  • OWASP Top 10 for LLM: LLM09
  • NIST AI RMF: MEASURE 2.11, MANAGE 2.4