In the past 2 years, there has been a surge in GenAI and its security implications. I developed the following framework to categorize the blogs, tools, and resources I have been exploring to make it easier to navigate and understand.
Everything related to GenAI and Security can be put in these 3 pillars:
- Security of GenAI
- GenAI for Security
- Security from Generative AI Threats
Security of GenAI
This refers to the security and robustness of the generative AI models themselves including the confidentiality, integrity, and availability of the AI models, training data, and inference processes. This includes protecting against attacks that could compromise the security of the generative AI system, such as model extraction, data poisoning, or adversarial inputs.
The goal is to ensure the generative AI systems can be safely and reliably deployed without being vulnerable to malicious exploitation.
Guidance such as OWASP Top 10 and MITRE Atlas are excellent examples of enabling security of GenAI.
GenAI for Security
This refers to the use of generative AI techniques to enhance or support security applications and capabilities. The example of this include using generative models to create synthetic data for security training, generating realistic test cases, automating certain security tasks like malware detection or incident response. Think of this as using GenAI for security tasks such as SOC analysts or threat modeling.
Security from Generative AI Threats
This refers to the potential security risks and threats posed by the misuse or malicious use of generative AI technologies. This include the potential for generative AI to be used for creating deepfakes, automating phishing or social engineering attacks, generating malicious code or content, etc. Addressing these threats requires developing safeguards, detection mechanisms, and mitigation strategies to protect against the malicious use of generative AI.
In summary, there are three aspects to generative AI security:
- Security of GenAI focuses on securing the generative AI systems themselves. Think OWASP Top 10 or MITRE Atlas.
- GenAI for security explores how generative AI technologies can be leveraged to enhance security capabilities. Think Level 1 SOC analyst capabilities, threat modeling, etc.
- Security from generative AI threats deals with mitigating the potential security risks and harms that can arise from the misuse of generative AI. Think deepfakes, diligently crafted social engineering attacks, etc.