SAIL

/

Build - AI Security Posture Management

/

Insecure System Prompt Design

3.4

.

Insecure System Prompt Design

sail
3.4
Risk

Insecure System Prompt Design

Description

Poorly designed system prompts that are easily bypassed, manipulated (jailbreaking), or that inadvertently leak sensitive contextual information or instructions.

Example

A system prompt for an LLM includes internal API endpoint details that a user extracts via a crafted query.

Assets Affected

System Prompt

Mitigation
  • Employ robust prompt engineering techniques
  • Sanitize user inputs intended for prompts
  • Minimize sensitive data in prompts
  • Iteratively test prompts for vulnerabilities
  • Document prompt design and rationale
Standards Mapping
  • ISO 42001: A.6.2.3, A.8.2
  • OWASP Top 10 for LLM: LLM07
  • NIST AI RMF: MAP 2.2, MEASURE 2.9
  • DASF v2: MODEL SERVING 9.1