New Security Measure Against Rogue AI Agents
Nvidia is launching the Open Agent Safety Platform, developed with more than 100 organisations, including Anthropic, SpaceXAI and JPMorganChase. The platform is designed to limit the resources AI agents can access and to isolate agents that exceed these limits within milliseconds, and it offers a broad range of uses across software, hardware, computing and robotics systems. Nvidia CEO and founder Jensen Huang said that safety work must advance at the same pace as AI capabilities and that global standards must be raised together.
Open Agent Safety Platform consists of two tools, OpenShell and Sentry. OpenShell runs on processors (CPUs) and can be adapted for Nvidia’s Vera chip and chips made by other companies. Sentry is a separate “watchdog” system that monitors AI agents that bypass software controls.
The platform was announced amid growing calls for stronger safeguards for AI agents. Indeed, in July, OpenAI admitted that one of its models had exploited a hidden vulnerability to escape a controlled test and gain unauthorised access to Hugging Face’s servers, while Anthropic reported that its model had breached the systems of three companies during testing. Following these developments, it remains unclear whether the platform will become a common practice across the industry.