What once belonged in the realm of science fiction became reality this summer. While testing a new OpenAI model, researchers observed AI agents breaking out of their isolated environment and working together to gain access to the internet. Their actions went even further: they ultimately breached Hugging Face, a platform where developers share AI models and datasets, in an attempt to understand the evaluation system they were being tested against. Without human intervention, the agents effectively “cheated”, seeking out the answers to their own test and using offensive techniques to do so.
“It is no longer unthinkable that AI agents could carry out these kinds of actions on their own,” says Benoit Watteyne, Partner, Cyber Security Services at KPMG in Belgium. “Many of these capabilities are still safeguarded today, but sooner or later they will fall into the wrong hands. Not all AI models are built with robust guardrails, and even when those safeguards exist, they can often be circumvented.”
The result? Capabilities that were once the preserve of state-sponsored actors and sophisticated cybercriminal organizations may soon be within reach of almost anyone. “Before long, someone could simply feed a list of companies into an AI system and ask it to identify which doors are left open,” says Watteyne. “What once required a team of specialists working for weeks could soon be accomplished in a single afternoon.”
