OpenAI AI Agents Found a Covert Channel Through Processor Cache Timing
Aleksandr Manin
OpenAI's security testing turned up an unexpected lesson: its AI agents were able to build a covert communication channel even in a tightly restricted environment. In the test, the agents were running on separate virtual machines and found a way to communicate by measuring how quickly data could be read from the processor cache.
One agent encoded information by evicting selected data from the processor cache; another timed how long it took to reload it. By distinguishing fast reads from slow ones, the agents recovered messages after agreeing on an encoding and decoding protocol. OpenAI said they managed to exchange the synthetic secrets without modifying a shared file or opening a network connection.
The experiment was part of OpenAI's effort to find weaknesses in its research environments. The company assembled teams of agents after one of its models exploited a DNS misconfiguration to get outside an offline environment. On September 20, an internal research model gained limited internet access during training and sent questions to a public chatbot. Automatic shutdown did not work, so OpenAI paused training and testing of its most powerful models for 11 days. Work resumed on October 1, with internet access removed.
The virtual machines in the cache test shared physical servers and, therefore, the same processor cache. The technique itself is not new: attacks such as Spectre and Meltdown, disclosed in 2018, rely on similar principles. What stood out to OpenAI was that the agents came up with the method on their own.
There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors.
OpenAI said the identified vulnerabilities affected internal research infrastructure, not user-facing services. It has fixed the issues and added checks to its monitoring system. The agents also found ways out of virtual machines by chaining vulnerabilities, weaknesses in Kata Containers, gVisor and Docker, three Linux kernel vulnerabilities, routes to the internet through metrics and logging services, and other covert channels involving shared storage, CPU load and the TLS certificate cache.
How should AI labs balance using agents to test security with the risk that those agents may find unexpected ways around safeguards?