I’m brought into two types of AI security conversations pretty much every day. One is about enabling the SOC and other areas of business with AI capabilities. The other conversation is about the risk of AI. This includes concerns of using shadow AI and all of the associated concerns including data loss, exposer to vulnerabilities and so on.
While high-level discussions of AI risk are common, many people underestimate how these risks can directly impact them. To make the issue more tangible, it’s important to provide specific examples of how Shadow AI can lead to breaches or data leaks.
One such example is the EchoLeak vulnerabilities, which demonstrated how attackers could exploit weaknesses in large language models (LLMs). A more frequent concern I hear is the risk of sensitive data exposure—either through users prompting LLMs with confidential information or by training agents on sensitive datasets.
For LLMs, the danger lies in users inadvertently sharing sensitive data in prompts or asking the model to reason over confidential information. For AI agents, the risk is in granting them access to sensitive systems or training them on data that should remain private.
The Register has reported on how LLMs can be used for data theft. The concern is that as users become more comfortable with AI tools, they may gradually expose sensitive data. Threat actors could then prompt these models to retrieve that information.
Recommendations for Mitigating Shadow AI Risks
To defend against these threats, consider the following:
- Access Control Maturity
AI will quickly expose weak access control or poor implementation of zero trust principles. - Data Security Practices
Evaluate how well your organization protects data. AI will highlight gaps in what data is accessible and what is used for training. - AI Evaluation Methods
Do you have a way to assess the AI tools you use? Look for signs of vulnerability, and ensure developers follow responsible AI principles. Techniques like Retrieval-Augmented Generation (RAG) allow AI to use data without training on it—an important safeguard. - Visibility and Governance
Can you see all AI tools in use across your organization? Are you able to block unauthorized tools while offering approved alternatives? Do you have an exception policy in place?
These are some of the key considerations when addressing the risks of Shadow AI. I’ll be teaching a course on this topic later this month through Pearson—you can find more details on that [HERE].