Pillar and Portkey Join Forces to Enhance Security for AI Applications

Ziv Karliner

and

August 15, 2024

min read

We are excited to announce our strategic integration with Portkey, a partnership aimed at bolstering security for AI applications in production environments.

Portkey, a frontrunner in open-source AI gateways, handles billions of LLM tokens daily for AI teams worldwide. By offering support for over 200 LLMs through a unified API, Portkey is revolutionizing the management and deployment of AI applications.

Pillar is dedicated to helping innovative companies adopt AI securely and efficiently. Our advanced detection and evaluation models are continuously refined through real-world interaction analysis, having monitored over 1,700 AI chatbots and 300,000+ conversations across various languages and use cases.

‍

This integration seamlessly incorporates Pillar's low-latency, model-agnostic security layer into Portkey's guardrails platform. With minimal setup, teams can now implement enterprise-grade security measures for their GenAI applications, fortifying them against both existing and emerging AI-specific threats.

Key Features:

Intelligent Runtime Protection: Leveraging threat intelligence, we detect and block adversarial attacks in real-time, safeguarding end-users, data integrity, and application performance.
Holistic Threat Scanning: Our comprehensive approach scrutinizes all application components, including prompts, models, meta-prompts, tools, and outputs, for a wide array of potential threats.
Advanced Data Protection: We identify over 40 distinct data types and categories, including sensitive information like PII and secrets, while also flagging toxic language and concealed unicode characters.
Alignment with Industry Standards: Our risk detection protocols align with leading AI security frameworks, including the OWASP Top 10 for LLMs and MITRE ATLAS, ensuring comprehensive protection.

This partnership marks a significant step forward in securing the future of AI applications. By combining Portkey's expansive LLM support with Pillar's cutting-edge security features, we're setting a new standard for safe and efficient AI deployment.

3 steps to integrate:

Add Pillar’s API key to Portkey
Create Guardrail Checks by selecting the Pillar evaluators you want
Set up actions on the Guardrails and then add the Guardrail to a request Config.

Portkey Documentation:

https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks/pillar

FAQs

What does the Pillar and Portkey integration do for AI applications in production?

The integration embeds Pillar's low-latency, model-agnostic security layer directly into Portkey's guardrails platform, enabling enterprise-grade protection for GenAI applications with minimal setup. Teams can defend against both known and emerging AI-specific threats across prompts, models, meta-prompts, tools, and outputs in real-time production environments.

How many LLMs does Portkey support and why does that matter for this security integration?

Portkey supports over 200 LLMs through a unified API and processes billions of LLM tokens daily. Because Pillar's security layer is model-agnostic, the integration extends consistent runtime threat detection and data protection across all supported models, giving AI teams broad coverage without model-specific configuration overhead.

What types of data threats can Pillar detect within the Portkey guardrails integration?

Pillar identifies over 40 distinct data types and categories, including personally identifiable information, secrets, toxic language, and concealed unicode characters. This data protection layer scans all application components — prompts, outputs, and tooling — ensuring sensitive information is flagged before it can be exposed or exploited.

How does Pillar's threat detection align with established AI security frameworks like OWASP and MITRE ATLAS?

Pillar's risk detection protocols are built to align with the OWASP Top 10 for LLMs and MITRE ATLAS, two leading AI security frameworks. This means threat coverage maps to industry-recognized attack categories, giving security and AppSec teams a structured, standards-compliant foundation rather than ad hoc detection rules.

How do you set up Pillar security checks inside Portkey in three steps?

First, add Pillar's API key to your Portkey account. Second, create Guardrail Checks by selecting the specific Pillar evaluators you want to apply. Third, configure the actions triggered by those guardrails and attach the Guardrail to a request configuration. Full setup details are available in Portkey's official documentation.