1. Plan: AI Policy & Safe Experimentation
1.1
Inadequate AI Policy
1.10
Incomplete Threat Modeling for AI Systems
1.2
Governance Misalignment
1.3
Inadequate Compliance Mapping
1.4
Undefined Risk Tolerance & Categorization
1.5
Unmonitored AI experimentation
1.6
Insecure Experiment Logging & Monitoring
1.7
Overly Permissive Permissions in Experimentation
1.8
Experiment Output Data Leakage
1.9
Unauthorized / Prohibited Component Usage
2. Code/No Code: AI Asset Discovery
2.1
Incomplete Asset Inventory
2.2
Shadow AI Deployment
2.3
Unidentified Third-Party AI Integrations
2.4
Undocumented Data Flows and Lineage
2.5
Lack of Clarity on AI System Purpose and Criticality
2.6
Overlooked Embedded or Inherited AI Functionality
2.7
Discovery of Outdated or Orphaned AI Assets
3. Build: AI Security Posture Management
3.1
Data Poisoning and Integrity Issues
3.10
Unvetted Use of Open-Source and Third-Party AI Components
3.11
Exposed or Hardcoded Credentials in Build Artifacts
3.12
Failure to Specify or Enforce Secure Model Requirements
3.13
Insufficient Understanding of AI System Boundaries
3.14
Exposed AI Access Credentials in Discovered Assets
3.2
Model Backdoor Insertion or Tampering
3.3
Vulnerable AI Frameworks and Libraries
3.4
Insecure System Prompt Design
3.5
Insecure ML & Data Pipeline Jobs
3.6
Intellectual Property (IP) Theft of Models
3.7
Misclassified or Undocumented Sensitive Data Usage
3.8
Insufficient Human Oversight in Model Development
3.9
Insecure Temporary Artifacts or Intermediate Data Storage
4. Test: AI Red Teaming
4.1
Untested Model
4.2
Incomplete Red-Team Coverage
4.3
Lack of Risk Assessment Process
4.4
Missing Documented Evidence of Red Teaming/Risk Assessment
4.5
Outdated Risk Assessment
4.6
Insecure Storage of Red Teaming Artifacts
4.7
Insufficient Multimodal Security Testing
4.8
Limited Foreign Language Red Teaming
4.9
Limited Scope of Evasion Technique Testing
5. Deploy: Runtime Guardrails
5.1
Insecure API Endpoint Configuration
5.10
Insecure Memory & Logging
5.11
Denial-of-Service (Resource Exhaustion)
5.12
Resource Abuse
5.13
Malicious Content Generation
5.14
Autonomous-Agent Misuse
5.15
Insecure Plugin/Tool Integration
5.16
Cross-Domain Prompt Injection (XPIA)
5.17
Policy-Violating Output
5.2
Unauthorized System Prompt Update/Tampering
5.3
Direct Prompt Injection
5.4
System Prompt Leakage
5.5
Context-Window Overwrite/Manipulation
5.6
Sensitive Data Leakage
5.7
Insecure Output Handling
5.8
Adversarial Evasion
5.9
Model Theft / Extraction
6. Operate: Safe Execution Environment (Sandbox)
6.1
Autonomous Code Execution Abuse
6.10
Autonomous Policy/Compliance Violation
6.2
Unrestricted API/Tool Invocation
6.3
Dynamic/On-the-Fly Dependency Injection
6.4
Task Decomposition for Policy Evasion
6.5
Indirect Prompt/Instruction Injection
6.6
Autonomous Resource Provisioning/Abuse
6.7
Cross-Agent/Inter-Agent Abuse
6.8
Agentic System Self-Modification
6.9
Covert Channel Use/Evasion
7. Monitor: AI Activity Tracing
7.1
Insufficient AI Interaction Logging
7.2
Missing Real-time Security Alerts
7.3
Undetected Model Drift/Degradation
7.4
Inadequate AI Audit Trails
7.5
Data Exfiltration via Monitoring/Telemetry
7.6
Absence of AI-Specific Incident Response Plan

SAIL

/

Operate - Safe Execution Environment - Sandbox

/

Agentic System Self-Modification

6.8

.

Agentic System Self-Modification

sail
6.8
Download
Risk

Agentic System Self-Modification

Description

Agent modifies its own source code, configuration, or operational memory to alter behavior, evade controls, or persist malicious changes.

Example

Agent rewrites its own code to disable logging or sandbox checks during runtime.

Assets Affected

Agentic platform (no code)

Model files

Agent Memory/cache

Mitigation
  • Write-protect agent code/config
  • Use integrity verification and versioning
  • Block self-modification at runtime
  • Audit all changes to code/config and require approval
Standards Mapping
  • ISO 42001: A.6.2.6, A.9.3
  • OWASP Top 10 for LLM: LLM06
  • NIST AI RMF: MANAGE 2.4, MEASURE 2.4