I’ve had a few asks about how to secure LLM applications. There are different ways to look at this question. There are tools available, which this article hits this topic here Securing Multi-Cloud Gen AI workloads using Azure Native Solutions | Microsoft Community Hub. This article follows the kill chain against the following architecture providing defense in depth defense tool recommendation capabilities.
Key technologies to consider are the following
- A Web Application Firewall – Your first line of defense against various injection type attacks. An example of this tool is Microsoft Azure WAF
- API Defense – Defense against risky APIs or APIs leaking data. An example of this tool is Microsoft Defender for API
- Container security – Defense against attack the running application. A example of this tool is Defender for Containers
- AI Prompt defense – Defense against prompt injection, jailbreaking, etc. An example of this tool is Defender for AI and prompt Sheild.
- DevOps – Looking at how your LLMs are deployed and proactively identify vulnerabilities. Microsoft Defender CSPM features is an example of this type of tool
- XDR platform – Monitoring alerts from your tools so you can respond if something goes wrong. Microsoft Sentinel is an example of this type of tool.
There are also known vulnerabilities, which the OWASP Top 10 for LLM is a popular resource to check against found HERE . Here is a short explanation of what this is for those that don’t know.
The OWASP Top 10 for Large Language Model Applications started in 2023 as a community-driven effort to highlight and address security issues specific to AI applications. Since then, the technology has continued to spread across industries and applications, and so have the associated risks. As LLMs are embedded more deeply in everything from customer interactions to internal operations, developers and security professionals are discovering new vulnerabilities—and ways to counter them.
From a monitoring viewpoint, sometimes I’m asked about alerting to consider for Microsoft Defender technology. Here is a article on that topic found HERE.
Hopefully these resources help you think about LLM security.