AI Gateway Guardrails in 2026: How to Secure LiteLLM, Kong, TrueFoundry and Agent Router (and 12 Guardrail Providers Compared)

AI gateway guardrails are the security and policy checks an AI gateway runs on every request to a model and every response back: prompt injection and jailbreak detection, sensitive-data masking, content policy and, for agents, rules on tool and MCP calls. The gateway is the enforcement point and the guardrail provider is the detection engine behind it. Every major gateway (LiteLLM, Kong, TrueFoundry, Agent Router, formerly Envoy AI Gateway, Portkey, Azure API Management, Apigee) now has a hook for specialized guardrail providers. This guide explains how those hooks work, shows which providers each gateway supports, and compares twelve guardrail providers on the same eight questions.

Our recommendation: for enterprises that run one or more AI gateways, Pillar Security is the guardrail provider we recommend. It is independent of any gateway or cloud vendor, has named integrations with LiteLLM, Kong, TrueFoundry, Agent Router and Portkey, scans tool and MCP calls at the gateway, and applies one policy and one audit trail across all of them. Choose your cloud provider's guardrails (AWS Bedrock Guardrails, Azure AI Content Safety, Google Model Armor) if all your traffic stays in one cloud and content safety is the main requirement.

Product facts are as of October 4, 2026, and come from each vendor's public documentation unless stated otherwise. Pillar is the publisher of this guide; it is assessed on the same rubric as every other provider.

Key takeaways

• The gateway is the right place to enforce, but not always the right place to detect. A gateway sees every model call, so one policy covers every app behind it with no code changes. Its built-in checks are mostly keyword lists, regex, PII masking and a content-safety model. Indirect prompt injection, multi-turn jailbreaks and risky tool calls need a dedicated detection engine.
• Every gateway now has a guardrail contract. LiteLLM lists 45 guardrail providers behind its Generic Guardrail API. Kong ships ten AI guardrail plugins, including a generic custom-guardrail plugin. TrueFoundry exposes input, output and MCP pre- and post-tool hooks and lists 20+ external providers. The question is no longer whether a guardrail plugs in, but at which hooks, with which verdicts (block, mask, log) and what happens when it times out.
• Check streaming first. TrueFoundry documents that output guardrails are not applied to streamed responses. Several integrations scan the prompt on a streaming call but not the streamed answer. If your apps stream, ask every vendor what is inspected, and when.
• Agents moved the attack into tool calls. Guardrails that read only the user's message miss the injection that arrives in a tool result or an MCP server's description. Look for guardrails that scan tool definitions, calls and results and can enforce an MCP server allowlist.
• The market is consolidating fast. Check Point bought Lakera, F5 bought CalypsoAI and CrowdStrike bought Pangea in 2025. In May 2026 Palo Alto Networks announced its acquisition of Portkey, which now calls itself the Prisma AIRS AI Gateway. A guardrail tied to one gateway or one security suite is a different bet from a gateway-neutral one.
• Most enterprises run more than one gateway. A platform team on LiteLLM, an API team on Kong or Apigee, a data team on Databricks. A guardrail provider that works across all of them gives one policy and one audit trail instead of three.

What are guardrails for AI gateways?

Guardrails for AI gateways are checks that run inside or beside the gateway on each prompt, response and tool call, and return a verdict (pass, block or mask) that the gateway enforces.

An AI gateway is a proxy between applications and AI models. Gartner describes it as an intermediary that gives a central control plane to secure, govern and observe AI workloads, with four jobs: routing, security (authentication, keys and input and output guardrails), cost control and observability.

A guardrail is one check on the content passing through: does this prompt contain an injection, does this response leak a card number, is this tool call allowed? It returns pass, block or mutate (for example, mask the PII and let the request continue), and the gateway acts on it.

The two are complementary. Pillar made this case in 2025 in Securing your AI via AI gateways: gateways are built for centralized management and resource control, not to catch threats native to AI workflows. What changed in 2026 is that the gateways built standard hooks for exactly this pairing.

Is an AI gateway enough to secure LLM applications?

No. A gateway is the right enforcement point, but on its own it is blind to five things.

Why enforce at the gateway:

1. One policy, every app. Applications behind the gateway inherit the policy without a code change, and a security team can switch a guardrail from monitor to block without a redeploy.
2. Inputs and outputs in one place. The gateway sees the full prompt, including retrieved context the application added, and the full response.
3. Identity and attribution. The gateway knows which application, team or tenant made the call, so findings can be scoped and routed.

Where the gateway is blind:

1. Streamed output. A gateway can only judge a response it has seen. Buffering the response adds latency; scanning chunks asynchronously means the first tokens can reach the user before a verdict.
2. Tool execution that never crosses the gateway. If an agent calls an MCP server or internal API directly, the gateway sees the model's request to call the tool and the result on the next turn, not the side effect.
3. Agents on laptops. Coding agents such as Claude Code and Cursor call their own model endpoints from developer machines, usually outside the enterprise gateway. See Best AI Coding Agent Security Tools.
4. Context across turns. A gateway sees requests one at a time. Multi-turn jailbreaks and slow exfiltration need a guardrail that keeps session state.
5. Excluded content. Some gateways skip the system prompt by default; TrueFoundry documents this.

How do specialized guardrail vendors integrate with AI gateways?

Guardrail vendors integrate with AI gateways in four ways: a plugin the gateway vendor ships, the gateway's generic guardrail API, a module the guardrail vendor ships to run inside the gateway, or a direct API call from the application. Which one applies to your gateway decides what the guardrail sees and what it can do.

A fifth, weaker pattern is out-of-band log ingestion: the provider reads model invocation logs after the fact. It supports audit and detection but cannot block.

Three settings decide how any integration behaves in production:

• Hook points: input only; input and output; a parallel "during call" scan that saves latency but cannot block the model call; or pre- and post-tool hooks for MCP.
• Verdict types: block only, or block plus mutate (mask, redact). TrueFoundry, for example, applies masking only in its mutate mode.
• Failure policy and timeout: TrueFoundry enforces a 5-second timeout on custom guardrails; the Pillar integrations for Kong and Agent Router expose fail-open and fail-closed settings. Decide which applications must fail closed before go-live.

LiteLLM guardrails

LiteLLM supports guardrails through its guardrails config block: a few built-in checks plus 45 third-party providers, run before the model call (pre_call), in parallel with it (during_call) or after it (post_call).

Built in: PII and PHI masking with Microsoft Presidio, in-memory prompt-injection detection, secret detection and redaction (enterprise), a tool-permission guardrail, sensitive-data routing to an on-premises model, and LLM-as-a-judge.

Third-party providers: LiteLLM's documentation lists 45, including Pillar Security, Lakera, Palo Alto Networks Prisma AIRS, CrowdStrike AIDR, Pangea, Zscaler AI Guard, AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, Guardrails AI, Noma Security, Lasso Security, HiddenLayer, Straiker, Aim Security, IBM and Microsoft Purview. Vendors without a dedicated integration can implement LiteLLM's Generic Guardrail API.

How to add Pillar to LiteLLM:

guardrails:
  - guardrail_name: pillar-security
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: https://api.pillar.security/api/v1/integrations/litellm
      api_key: os.environ/PILLAR_API_KEY
      default_on: true
      additional_provider_specific_params:
        plr_mask: true       # mask PII, card data and secrets before they reach the model
        plr_evidence: true   # return what was detected and where
        plr_scanners: true   # return per-guardrail verdicts

Tools and MCP servers in each request are registered in Pillar's AI inventory and their arguments and results are scanned. Through LiteLLM, Pillar also enforces an MCP server allowlist and per-server tool-call permissions. A request header (x-plr-app-id) attributes traffic to an application or tenant, so one LiteLLM proxy can carry different policies for different apps. OpenWebUI and n8n deployments that route through LiteLLM inherit the same guardrails.

Watch for: during_call saves latency but cannot stop the model call, so use it for monitoring, not blocking.

Kong guardrails (Kong AI Gateway)

Kong AI Gateway supports guardrails through plugins: four native ones, first-party plugins for five guardrail providers, and an AI Custom Guardrail plugin for everything else.

Built in: AI Prompt Guard (allow and deny lists), AI Semantic Prompt Guard and AI Semantic Response Guard (meaning-based matching), and AI LLM as Judge.

Third-party providers: first-party plugins for AWS Bedrock Guardrails, Azure AI Content Safety, Google Cloud Model Armor (request side only), Lakera Guard and NVIDIA NeMo Guardrails, plus the AI Custom Guardrail plugin. Vendors also ship their own Kong plugins, among them Pillar and CrowdStrike AIDR.

How Pillar works on Kong: a Pillar plugin for Kong 3.9+ with the AI Proxy scans prompts on the way in, responses on the way out, and tool definitions, tool calls and tool results. A buffered variant holds the full response until it is scanned and blocks it if flagged; a streaming variant passes server-sent events through in real time and scans them asynchronously. Settings (scan input, scan output, scan tools, timeout, fail-closed and a session header for multi-turn tracking) are managed in Kong Manager, the Admin API or declarative config, with the API key in a Kong Vault. On Kong 3.4+ without the AI Proxy, a pre-function script gives input blocking and log-only output scanning.

Watch for: Kong's Model Armor plugin covers requests only. Buffered output scanning trades latency for a guaranteed block.

TrueFoundry guardrails

TrueFoundry AI Gateway runs guardrails at four hooks (LLM input, LLM output, MCP pre-tool and MCP post-tool), with built-in guardrails and 20+ external providers, including Pillar.

Built in: content moderation, PII and PHI detection and redaction, and prompt-injection detection (SaaS only); secrets detection, a code-safety linter, a SQL sanitizer, regex matching, and Cedar and OPA policy guardrails (also self-hosted).

Third-party providers: Pillar Security, Palo Alto Networks Prisma AIRS, CrowdStrike, Cisco AI Defense, F5 AI Security, AWS Bedrock Guardrails, Azure Content Safety and Prompt Shield, Google Model Armor, NVIDIA NeMo, Guardrails AI, Lasso Security, Patronus AI, Gray Swan, Enkrypt AI, Fiddler, Arthur AI, DeepKeep, TrojAI and OpenAI Moderations, plus a template for custom guardrails.

How it works: each integration runs in validate mode (parallel; pass or block) or mutate mode (sequential; can rewrite content), with an enforcing strategy of enforce, enforce but ignore on error or audit.

How to add Pillar to TrueFoundry: register two custom guardrail integrations, one on the input hook and one on the output hook, then bind them to models with guardrail rules. Use mutate so Pillar masks PII, card data and secrets in flight instead of blocking the request.

Watch for: output guardrails are not applied to streamed responses, custom guardrails time out after 5 seconds, and system prompts are excluded by default.

Agent Router (formerly Envoy AI Gateway) guardrails

Agent Router, the open-source Kubernetes AI gateway formerly called Envoy AI Gateway, has no built-in guardrails; they run as Envoy filters or external-processing services, and Pillar ships a native module for it.

The project was renamed in September 2026 and is now an Agentic AI Foundation project. Deployed resources are unchanged: the AIGatewayRoute and AIServiceBackend resources, the aigateway.envoyproxy.io API group, the aigw CLI and the Helm charts keep their names, so existing guardrail configurations keep working.

How Pillar works on Agent Router: an Envoy dynamic module, attached to the gateway with an EnvoyExtensionPolicy. It scans prompts, completions or both; recognizes the OpenAI Chat Completions, OpenAI Responses and Anthropic Messages formats whichever backend a route targets; rejects flagged traffic with a 403; and fails closed by default if Pillar is unreachable. Non-LLM paths such as embeddings pass through untouched. On streaming calls the prompt is scanned before the model is reached; scanning of streamed responses is planned.

Portkey guardrails (now the Prisma AIRS AI Gateway)

Portkey runs guardrails as before_request_hooks and after_request_hooks, with 20+ built-in checks and partner integrations including Pillar, Lakera and Prompt Security; it is now owned by Palo Alto Networks.

Built in: deterministic checks (regex, JSON schema, code detection, length limits) and LLM-based checks for gibberish and prompt injection.

Third-party providers: partners include Pillar Security, Lakera, Prompt Security, Patronus, Lasso, Pangea, Aporia, Azure, Bedrock and F5. Pillar's Portkey integration offers Scan Prompt (prompt injection, PII, secrets, toxic language, invisible characters) and Scan Response (PII, secrets, toxic language).

Watch for: Palo Alto Networks announced its acquisition of Portkey in May 2026, and Portkey's documentation now says it "is now Prisma AIRS AI Gateway." If you run Portkey with a guardrail from another vendor, confirm that partner's roadmap with your account team.

Guardrails in other AI gateways

Which guardrail providers work with which AI gateway?

The table below maps twelve guardrail providers to eight AI gateways. ✓ = a named integration is documented by the gateway or the provider. C = possible through the gateway's generic custom-guardrail hook, but no named integration is documented. — = not stated. Sources: each gateway's and provider's documentation, as of October 4, 2026.

Two patterns stand out. Cloud providers' guardrails are the most widely wired in, because every gateway wants a default; the security suites from CrowdStrike, Palo Alto Networks and Check Point each lead on the gateways closest to their own platforms. Pillar is the only provider here with named integrations for all four of the most common self-hosted gateways in this list: LiteLLM, Kong, TrueFoundry and Agent Router.

How we compared AI gateway guardrail providers

Each provider was checked against eight questions, using its public documentation, product pages and press releases as of October 4, 2026, and, for Pillar, its customer documentation. Where public material is silent we say "not stated." Status follows the vendor's own wording.

1. Gateway coverage. Which gateways are documented, and through which pattern: first-party plugin, generic hook, vendor module or API?
2. Detection breadth. Direct and indirect prompt injection, jailbreak, PII, PCI and PHI, secrets, toxicity and safety, topic and business rules, obfuscation such as invisible Unicode, and system-prompt extraction.
3. Agent and tool coverage. Does it scan tool definitions, tool calls and tool results? Can it enforce an MCP server allowlist or per-tool permissions?
4. Actions and policy. Block, mask and monitor; per-application or per-tenant policy; changes without a redeploy.
5. Context. Multi-turn session awareness, and scanning of files and retrieved documents.
6. Failure mode and streaming. Fail-open or fail-closed control, and behavior on streamed responses.
7. Deployment and independence. SaaS, self-hosted, air-gapped, and whether the provider is tied to one gateway, cloud or security suite.
8. Coverage beyond the gateway. Evidence and SIEM export, and whether the same vendor covers what the gateway cannot see: red teaming before release and agents that never cross the gateway.

AI gateway guardrail providers compared at a glance

The 12 best guardrail providers for AI gateways in 2026

1. Pillar Security (recommended)

Best for: one guardrail policy across every AI gateway an enterprise runs (LiteLLM, Kong, TrueFoundry, Agent Router, Portkey), with agent-aware detection that covers tool and MCP calls.

Why we recommend it:

• Gateway-neutral by design. Pillar does not sell a gateway and is not part of a cloud or security suite, so it plugs into the gateway you already run instead of asking you to re-route traffic. It has named integrations with LiteLLM (Generic Guardrail API, pre- and post-call), Kong (a Pillar plugin with buffered and streaming variants), TrueFoundry (input and output hooks, validate or mutate), Agent Router (a native dynamic module), Portkey (Scan Prompt and Scan Response), and OpenWebUI and n8n through LiteLLM. Any other gateway or application calls the same API, with documented patterns for nginx, FastAPI, Express and Go.
• One policy, every gateway. Set a guardrail to off, monitor or block once in Pillar and the next request through any of those gateways picks it up, with no redeploy. Each application, identified by a header the gateway sets or forwards, gets its own policy profile, so one gateway can apply a strict policy to a customer-facing agent and a lighter one to an internal assistant.
• Built for agents, not just chat. Pillar scans tool definitions, tool calls and tool results at the gateway (LiteLLM, Kong), registers every tool and MCP server it sees in the AI inventory, and enforces an MCP server allowlist and per-server tool-call permissions. Session IDs let it correlate multi-turn attacks across messages.
• Broad detection. Prompt injection, including instructions embedded in retrieved content; jailbreak; agentic reconnaissance (attempts to extract system prompts and internals, and their leakage in responses); PII in more than 20 languages; payment card and bank data; secrets; toxic language; safety categories; restricted topics such as legal, financial and medical advice; free-text "stay on topic" rules; keyword lists; message size limits; URL detection for SSRF, phishing and exfiltration; and evasion such as invisible Unicode and bidirectional overrides, the technique behind Pillar's Rules File Backdoor research. Masking replaces PII, card data and secrets with placeholders before the prompt reaches the model, while the original is kept in Pillar's audit log. Files and attachments are scanned by the same guardrails.
• Covers what the gateway cannot see. The same platform discovers AI assets, red-teams applications before they ship, and secures coding agents on developer machines through Agentic Endpoint, the traffic that usually bypasses the enterprise gateway. Policies and findings map to the SAIL framework.
• Deploys where you need it. Managed SaaS or self-hosted on Kubernetes for air-gapped and data-residency requirements. Findings stream to SIEM tools such as Splunk and to Jira and Slack.

Where it is strongest: organizations running more than one gateway, agentic applications that call tools and MCP servers, and security teams that want to own guardrail policy without touching gateway configuration.

2. CrowdStrike Falcon AIDR (formerly Pangea)

Best for: Falcon customers who want AI detection and response in the same console as endpoint and cloud.

CrowdStrike acquired Pangea in 2025 to build AI Detection and Response (AIDR). It has broad gateway reach: Kong plugins, Apigee and Azure API Management integrations, and named provider listings in LiteLLM, TrueFoundry and Portkey (as Pangea). TrueFoundry notes it is the one provider that scans system prompts by default.

Limitations to check: whether policy and evidence live in Falcon or a separate console, tool and MCP enforcement at the gateway (its MCP proxy works on the client side), and the commercial tie to Falcon.

3. Check Point AI Security (Lakera Guard)

Best for: teams whose first concern is prompt injection, on Kong, LiteLLM or Portkey.

Lakera Guard screens prompts and responses through one API for prompt attacks, data leakage and content policy, and redacts PII spans. Kong ships a first-party AI Lakera Guard plugin, LiteLLM has a v2 integration and Portkey lists it as a partner. Check Point announced the acquisition in September 2025.

Limitations to check: TrueFoundry does not list Lakera among its named providers; confirm tool and MCP coverage per gateway and the roadmap inside Check Point.

4. Palo Alto Networks Prisma AIRS (and the Prisma AIRS AI Gateway)

Best for: Palo Alto Networks customers who want the gateway and the guardrail from one vendor.

Prisma AIRS runtime security inspects prompts, responses and agent activity and is a named provider in LiteLLM and TrueFoundry. Palo Alto Networks announced its acquisition of Portkey in May 2026 to make the AI gateway the control plane for Prisma AIRS.

Limitations to check: the gateway and the guardrail become one commercial decision. If you run Kong, Agent Router or another gateway, confirm the integration path.

5. AWS Bedrock Guardrails

Best for: AWS-centric estates.

Bedrock Guardrails offers content filters, denied topics, word filters, sensitive-information filters, contextual grounding checks and prompt-attack detection. Its ApplyGuardrail API checks content for any model, which is how LiteLLM, Kong, TrueFoundry, Portkey and agentgateway call it.

Limitations to check: policy lives in AWS accounts, so cross-cloud and cross-gateway consistency needs extra work; confirm agent and tool coverage outside AgentCore.

6. Azure AI Content Safety and Prompt Shields

Best for: Azure estates, especially behind API Management.

Content Safety moderates text and images across harm categories; Prompt Shields detects direct and indirect prompt injection. API Management applies it natively, and Kong, LiteLLM, TrueFoundry and Portkey integrate it.

Limitations to check: coverage beyond content safety, such as business rules, secrets and tool-level policy, outside the Microsoft stack.

7. Google Cloud Model Armor

Best for: Google Cloud estates, especially behind Apigee.

Model Armor screens prompts and responses for prompt injection, jailbreak, sensitive data, malicious URLs and responsible-AI categories. Apigee applies it natively, and LiteLLM, TrueFoundry and agentgateway integrate it. Kong's plugin covers requests only.

Limitations to check: response-side and tool-call coverage on gateways other than Apigee.

8. F5 AI Guardrails (formerly CalypsoAI)

Best for: F5 customers and model-agnostic inline policy in private environments.

F5 acquired CalypsoAI in 2025. Its guardrails are listed in TrueFoundry ("F5 AI Security") and Portkey, and F5 also sells its own AI gateway.

Limitations to check: named integrations with LiteLLM and Kong, and agent and tool controls.

9. Cisco AI Defense

Best for: organizations standardized on Cisco security.

Cisco AI Defense, built on the 2024 Robust Intelligence acquisition, provides runtime guardrails and model validation and is a named provider in TrueFoundry.

Limitations to check: named integrations with other gateways.

10. Zscaler AI Guard

Best for: Zscaler customers who want AI application guardrails next to the Zero Trust Exchange.

Zscaler AI Guard is a named LiteLLM guardrail provider.

Limitations to check: integrations with Kong, TrueFoundry and other gateways, and tool and MCP coverage.

11. NVIDIA NeMo Guardrails (open source)

Best for: teams that want programmable, self-managed rails.

NeMo Guardrails is an open-source toolkit for input, output, dialog, retrieval and execution rails. Kong ships a plugin for it and TrueFoundry lists it.

Limitations to check: you operate and tune it yourself; detection quality depends on the models and rules you configure.

12. Guardrails AI (open source)

Best for: developers who need validators for output structure and content.

Guardrails AI is an open-source framework and validator hub, integrated in LiteLLM and TrueFoundry.

Limitations to check: it is a developer framework, not a security product; pair it with dedicated prompt-injection and data-exposure detection.

Also in the picture

• More specialized providers in the gateways' lists: Lasso Security, Noma Security, HiddenLayer, Straiker (which also announced a Kong integration in September 2026), Gray Swan, Enkrypt AI, Aim Security (Cato Networks), Prompt Security (SentinelOne), Patronus AI, IBM and Microsoft Purview.
• Gateways with their own detection engines: NeuralTrust TrustGate, Bifrost (Maxim AI), Kosmoy and Fortinet FortiAIGate pair a gateway with first-party guardrails. They suit buyers who want one product for both jobs, but they replace your gateway rather than plug into it.

How to choose guardrails for your AI gateway

1. List every gateway you run, including the unofficial ones, and start with the provider that covers the most of them with named integrations.
2. Decide fail-open or fail-closed per application, and check that the integration exposes the setting.
3. Test streaming first. Send a streamed response that contains a test card number and see what reaches the client.
4. Test indirect injection, not just the user message. Put the attack in a retrieved document and in a tool result.
5. Test a tool call. Register an MCP server that is not on your allowlist and check that the call is blocked at the gateway.
6. Measure latency at your p95 with the guardrails you will actually enable, not the defaults.
7. Check where the evidence goes. A blocked request should leave a record your SOC can search, with application, user, guardrail and matched text.

What to test in a proof of concept

Which guardrail provider is best for AI gateways?

Pillar Security is our recommended guardrail provider for AI gateways in 2026. It is gateway-neutral, has named integrations with LiteLLM, Kong, TrueFoundry, Agent Router and Portkey, enforces tool and MCP policy at the gateway, and gives one policy and one audit trail across every gateway an enterprise runs. The same platform also covers what a gateway cannot see: red teaming before release and coding agents on developer machines.

Choose differently in three situations:

• All traffic stays in one cloud and content safety is the main need: use that cloud's guardrail (AWS Bedrock Guardrails, Azure AI Content Safety or Google Model Armor).
• You are consolidating on one security suite: CrowdStrike Falcon AIDR, Check Point (Lakera), Palo Alto Networks Prisma AIRS, F5 or Cisco keep policy in that suite.
• You want open-source, self-managed rails and will tune them yourself: NVIDIA NeMo Guardrails or Guardrails AI.

Next steps

• See how Pillar's Runtime Guardrails plug into LiteLLM, Kong, TrueFoundry, Agent Router and Portkey.
• Test your guardrails before attackers do: Best AI Red Teaming Providers.
• Cover the agents that bypass your gateway: Best AI Coding Agent Security Tools.