Attacking generative AI
Managing risks in large language models
Large language models (LLMs) such as GPT-3 and Copilot have become increasingly popular for efficieint translation, chatbots and content generation. However, as with any technology, this type of artificial intelligence can open up new attack surfaces.
Consider the following potential vulnerabilities in both the training phase and production phase.
1
Training phase
An LLM can only perform as well as it is trained to do, based on select datasets provided by AI engineers. These datasets create risk for:
- Data leakage or poisoning. LLM training data can be unintentionally exposed to tampering that leads to vulnerabilities or biases, which can compromise the security, effectiveness or ethics of the model.
- Model theft. While an LLM is in development, many people are typically given access to complete the model, which increases its susceptibility to theft by an insider threat actor. The development phase also increases the risk of other identity-related vulnerabilities.
- Insecure plug-in design. Without proper testing, third-party plug-ins for LLMs can introduce risks such as insecure authentication and authorization configurations. They can also allow for insecure inputs.
2
Production phase
Once an LLM has been deployed, users throughout an organization can typically access it for a wide array of purposes. Look out for risks such as:
- Prompt injection. Similar to SQL injections or cross-site scripting (XSS), an attacker could input payloads into an active LLM, causing it to perform unintended actions.
- Sensitive information disclosure. If an LLM isn’t properly trained and configured, an attacker could craft a prompt that causes the model to reveal sensitive information.
- Excessive agency. Often, LLMs are granted far more permissions than are required for them to accomplish their functions. Attackers can recognize and leverage these permissions to elevate their own privileges, causing harm to back-end systems.
3
A need for ongoing testing
To mitigate risks throughout the LLM lifecycle, consider the following high-level approach for security testing:
This kind of testing is an important part of cybersecurity, as threats to LLMs can result in additional attack vectors against related application programming interfaces (APIs) and networks. Conducting a thorough penetration test for LLM applications at each phase of the lifecycle can help you identify vulnerabilities, improve your security posture, and mitigate risks.
How KPMG can help
KPMG offers end-to-end security testing as an outcome-based managed service, helping you consistently validate controls while minimizing remediation efforts. That’s because business transformation is not a fixed destination; it’s an ongoing journey. With managed services, we help you continually evolve your business functions to keep up with ever-changing targets, while driving outcomes like cost reduction, resilience, and stakeholder trust.
Explore more
Insight
Social engineering
A fatal flaw in cybersecurity
Insight
Preventing broken trust
Managed services can help fill critical role of application security
Insight
Application security as a culture
How to counter agile adversaries
Meet our team
