Nvidia Builds a Safety Layer for AI Agents

AI agents are being given more freedom to use tools, access data and take actions on their own. Nvidia now wants to put a hard boundary around that autonomy. The company has launched the Nvidia Open Agent Safety Platform, an open-source software platform and reference system designed to monitor and control AI agents from testing through deployment. 

The launch follows several incidents in which autonomous AI systems accessed systems they were not supposed to reach, including the recent Hugging face ( an AI – ML community ) high-profile breach involving OpenAI agents.

The platform solution from NVIDIA has two main pieces. OpenShell creates a secure runtime boundary around an AI agent. It tracks what the agent is doing and enforces policies governing access to data, tools, applications and services. Importantly, Nvidia says OpenShell can also work with third-party compute platforms, including Arm and Intel hardware.

Then there is Sentry, the hardware-level watchdog. It runs on Nvidia BlueField-4 DPUs and operates outside the agent itself. If an agent attempts to cross an approved boundary, Sentry can quarantine it and stop the activity in milliseconds, according to Nvidia.

That external layer is the important part. Instead of relying entirely on an AI model to follow instructions, the controls sit outside the model and can restrict what the agent is technically allowed to do.

More than 100 organisations are working with Nvidia  and testing the platform