Leveraging Adversary Emulation for GenAI Red Teaming

Securing GenAI systems is a constantly evolving challenge that requires innovative security measures. Unlike traditional systems, GenAI responses are typically non-deterministic, making them challenging for traditional cybersecurity approaches to address. Commensurate security approaches are imperative to address this challenge.  Adversary emulation is an approach to security testing that enables realistic, non-deterministic testing based on actual attacker tactics, techniques, and procedures (TTPs). This talk presents strategies and practical examples for combining adversary emulation with security chaos engineering to address the security challenges posed by GenAI systems.

This talk explores the powerful synergy between adversary emulation and GenAI red teaming, offering a comprehensive approach to identifying and mitigating potential vulnerabilities in AI systems. The fundamentals of adversary emulation would be highlighted, illustrating the emulation of real-world attack scenarios that uncover system weaknesses. The discussion will then shift to the unique challenges of GenAI systems, including their non-deterministic nature, and how a combination of adversary emulation and security chaos engineering yields significant benefits.

The presentation's core hinges on demonstrating how adversary emulation can be effectively adapted for GenAI red teaming by leveraging MITRE ATT&CK and MITRE ATLAS, for example, data poisoning attacks, prompt injection, and data exfiltration.

By the end of this talk, attendees will understand how adversary emulation can enhance GenAI red teaming, ultimately leading to more secure and robust AI systems.