Risk
Malicious or Policy-Violating Action by Agent
Description
An agent generates harmful or policy-violating output, or takes a tool action, that breaches organizational, industry, or regulatory policy.
Example
An agent generates hate speech, provides unauthorized investment advice, or autonomously executes a transaction that violates compliance policy.
Assets Affected
Model Response
AI Agent
AI App
Tool
Mitigation
- Output filtering and content moderation
- action-policy enforcement at runtime
- HITL for high-risk actions
- compliance monitoring
- restrict high-risk use cases.
Standards Mapping
- ISO: A.5.4
- OWASP agentic: ASI02
- OWASP LLM: LLM09:2025
- EU AI Act: Art. 50, Art. 15
- DASF: Model Serving 9.8, Model Serving 10.2
- AIUC-1: C003, C004