AI Systems Must Be Secured By Design

Dor Sarig

and

January 30, 2024

min read

Recent blog by Cybersecurity and Infrastructure Security Agency illustrates the importance of embedding security in AI systems from the inception of the idea to the system's end-of-life.

Like any software system, we must prioritize security throughout the AI development lifecycle.

‍

Key Takeaways

‍

AI as Software

AI is a type of software system that generates predictions, recommendations, or decisions based on statistical reasoning. It should be treated with the same rigor as other software when it comes to security.

Secure by Design

AI systems should be secure by default. Security should be a core business requirement, not just a technical feature.

Security Best Practices

Despite the uniqueness of AI systems, the AI community should adopt existing security best practices and augment them where needed. Practices like vulnerability identification, using memory-safe languages, and respecting privacy principles should be instituted.

‍

AI-Specific Challenges

AI-specific assurance issues, such as adversarial inputs causing model jailbreaking or data leakage, require attention. Ensuring security in non-AI components of the system is equally important.

‍

AI's Critical Role

AI has become integral to various aspects of our society, automating processes that are crucial to our lives. As AI continues to grow, the importance of designing AI systems securely increases.Let's remember, security in AI systems is not an afterthought – it is a continuous practice. Secure by Design principles are a foundation for ensuring that AI systems remain safe, fair, and effective.

‍

FAQs

Why should AI systems be treated the same as traditional software from a security perspective?

AI systems are a type of software that generates predictions, recommendations, or decisions based on statistical reasoning. Because they share the fundamental nature of software systems, they require the same security rigor applied across any software lifecycle — from initial concept through end-of-life — not a separate or reduced standard.

What does 'Secure by Design' mean in the context of AI systems?

Secure by Design means security is embedded into AI systems from inception rather than added later. It treats security as a core business requirement, not merely a technical feature. This approach ensures that safety, fairness, and effectiveness are foundational properties of the system, sustained continuously across its entire lifecycle.

How should the AI community adapt existing cybersecurity best practices for AI-specific development?

The AI community should adopt established security best practices — such as vulnerability identification, use of memory-safe programming languages, and privacy-by-design principles — and augment them where AI introduces unique challenges. The goal is to extend proven security discipline into AI development rather than treating AI as an entirely separate domain requiring entirely new frameworks.

What AI-specific security threats require attention beyond standard software vulnerabilities?

AI systems face assurance challenges that go beyond conventional software vulnerabilities, including adversarial inputs that cause model jailbreaking and unintended data leakage. Securing non-AI components within the broader system is equally critical, as weaknesses in surrounding infrastructure can undermine the integrity of the AI model itself.

Why is securing AI systems becoming more urgent as AI adoption grows?

AI has become integral to automating processes that are crucial to society, meaning failures or compromises carry significant real-world consequences. As adoption accelerates, the attack surface expands and the stakes of insecure design increase proportionally — making Secure by Design principles not optional guidance, but a necessary foundation for responsible AI deployment.