Orca Security has unveiled AI Goat, the first open source AI security hands-on learning environment based on the OWASP Top 10 Machine Learning (ML) risks. Launched by the Orca Research Pod, AI Goat is an intentionally vulnerable AI environment built using Terraform and hosted on AWS, designed to help security professionals and penetration testers gain practical experience in identifying and mitigating AI-specific vulnerabilities.
A New Resource for AI Security Education
AI Goat provides a unique platform for security teams to explore how AI models can be exploited through various attack vectors outlined by the OWASP Top 10 ML risks. This hands-on learning environment includes scenarios such as Data Poisoning, AI Supply Chain Attacks, and Output Integrity Attacks, all within a simulated online store that sells soft toys. The tool is available as an open source project on Orca Research’s GitHub repository, allowing for broad accessibility and customization.
“Orca’s AI Goat is a valuable resource for AI engineers and security teams to learn more about the potentially dangerous misconfigurations and vulnerabilities that can exist when deploying AI models,” said Shain Singh, one of the OWASP ML Security Top 10 project leaders. “By using AI Goat, organizations can enhance their understanding of AI risks and the different ways attackers can leverage these weaknesses.”
Deploying AI Goat: An Accessible Learning Tool
Setting up AI Goat is streamlined through Terraform, making it easy for users to deploy the environment on AWS. The tool allows users to explore three main missions that demonstrate different AI vulnerabilities. For instance, in the AI Supply Chain Attack mission, users exploit the product search functionality to access sensitive files, while the Data Poisoning Attack mission challenges participants to manipulate a model’s recommendations.
To ensure that organizations can safely experiment without compromising their broader environments, Orca Security advises users to limit exposure to critical systems and to delete the AI Goat environment after use.
Showcasing at DefCon and Future Implications
Orca Security is set to demonstrate AI Goat at the DefCon Arsenal on August 9th. This session will provide attendees with a live walkthrough of AI Goat, highlighting how to deploy it, explore its vulnerabilities, and gain hands-on experience in AI security. The session promises to be an interactive opportunity for cybersecurity professionals to deepen their understanding of AI risks.
Melinda Marks, Practice Director of Cybersecurity at Enterprise Strategy Group, emphasized the importance of such tools, stating, “Orca’s AI Goat makes an important contribution to the community to help organizations gain hands-on experience to better understand possible threats and methods of attacking AI models so they can mitigate security risk and defend against possible attacks.”
Orca’s Commitment to AI Security
Beyond AI Goat, Orca Security continues to expand its AI Security Posture Management (AI-SPM) capabilities, offering organizations a comprehensive view of all AI models deployed within their environments. Orca’s platform provides continuous monitoring and risk assessment, alerting organizations to vulnerabilities, exposed data, and other security issues. Automated remediation options further empower teams to quickly address any identified risks.
