AI 技术NVIDIA Blog5/10

AI Leaders Propose SAFE Guidelines for Cybersecurity Transparency

By Justin Boitano

AI Leaders Propose SAFE Guidelines for Cybersecurity Transparency
图源:NVIDIA Blog

AI 摘要

Members of the Open Secure AI Alliance — now more than 120 organizations strong — are developing new guidelines to strengthen agentic AI cybersecurity as the annual Black Hat conference begins in Las Vegas today.  The Linux Foundation today shared a Request for Comments on Sh

原文正文

Members of the Open Secure AI Alliance — now more than 120 organizations strong — are developing new guidelines to strengthen agentic AI cybersecurity as the annual Black Hat conference begins in Las Vegas today.

The Linux Foundation today shared a Request for Comments on Shared AI Findings Exchange (SAFE), a proposed set of guidelines designed to turn agentic cybersecurity incidents into shared protection for the entire ecosystem.

The SAFE guidelines are being drafted by an Open Secure AI Alliance working group. NVIDIA, Cisco, CrowdStrike, Hugging Face and Red Hat are among Open Secure AI Alliance members working with the Linux Foundation to contribute to the initial proposal.

The SAFE guidelines include proposals to confidentially collect and analyze AI incidents and near misses, inform those impacted, identify recurring control failures and publish evidence-based operating recommendations that reduce systemic risk.

Cybersecurity is a race without a finish line. Every major technology shift has created new potential attack surfaces. Defenders must move now at agent speed to respond rapidly to protect infrastructure and intellectual property — and the best way to do that is together. When trusted ecosystems share threat intelligence openly, collective defense becomes a force multiplier.

Open Secure AI Alliance Delivers More Tools for AI Cybersecurity

The SAFE framework adds to technology contributions Open Secure AI Alliance members are making as part of a shared commitment to building and sharing open, inspectable tools across the full AI security stack.

An AI agent isn’t just a model. It’s a system — identity controls, harnesses, guardrails, logs and evaluation — and securing it requires more than vulnerability scanning.

Security has always been strongest in the layers — and in the community’s willingness to share what it knows. The hardest problems get solved when defenders learn from each other, openly and at speed.

Full Stack of Open NVIDIA Cybersecurity Software and Models

NVIDIA’s contributions run the length of the stack, starting with the NVIDIA Labs Object-Oriented Agent (NOOA) research harness, on GitHub — which makes agent behavior easier to test, trace, audit and govern.

The NVIDIA OpenShell runtime restricts what an agent can see, touch and do — enforcing security and privacy controls at the agent level, so an agent can’t reach what it shouldn’t.

NVIDIA’s open model families — NVIDIA Nemotron for agentic AI, NVIDIA Cosmos for physical AI, NVIDIA Isaac GR00T for robotics, NVIDIA BioNeMo for healthcare and life sciences, and NVIDIA Alpamayo, the world’s largest model for autonomous vehicles licensed for commercial use — ship with open weights, datasets and training techniques.

NVIDIA open source verified agent skills extend that trust to the capability layer.

Each skill provides portable instruction sets — cataloged, scanned for risks such as prompt injection and tools poisoning, cryptographically signed and documented with a skill card. Defenders know exactly what an agent skill does, where it came from and whether it was modified after publication.

NeMo Guardrails, NeMo Anonymizer and NeMo Safe Synthesizer help enforce safety policies, protect sensitive data and generate privacy-safe synthetic data.

And Garak, NVIDIA’s open source LLM vulnerability scanner, lets security teams check models for data leaks, prompt injections and jailbreak scenarios before they ship.

Alliance Members Expand Tools for Open Ecosystem Development

Other members of the Open Secure AI Alliance have also been building across the full defensive stack, spanning identity and permissions, harnesses, runtime guardrails, security AI models, observability and evaluation, data security and privacy, availability and resilience, and more.

Some of the latest contributions across different layers of the stack are highlighted below, with more continuing to arrive.

Identity and Permissions — Who Gets to Act

You can’t secure what you can’t identify.

Okta is developing reference implementations for agent identity and access, showing how Cross App Access (XAA), an open protocol, enables AI agents operating in OpenShell sandbox environments to securely connect to enterprise applications.

Palo Alto Networks has contributed open source tools from Idira, its next-generation identity security platform, including Agent Guard and Agent Watch. These tools help developers and agent builders apply identity security best practices and safe guards such as securely retrieving secrets for agentic workflows.

A new open source project founded by Red Hat, asago takes an organization’s custom governance requirements — such as those referenced in NIST, OWASP and the EU AI Act — and maps them directly to what agents are allowed to do at runtime, with a single audit trail from policy clause to live control.

Harnesses and Tooling — How Security Work Gets Orchestrated

AI agents are more than a model – they tap into systems of open and closed models, harnesses, tools and runtimes to get work done.

If a model is the agent’s brain, the harness is the body that takes action by using tools. The harness surrounding the model acts like an orchestrator that determines how agents are deployed, coordinated and constrained.

Alliance members are contributing tooling, harnesses and supporting technologies across this emerging layer of the AI security stack.

Amazon, which today became one of the newest members of the Open Secure AI Alliance, contributes Strands Agents, an open source toolkit for building AI agents that is open at every layer, giving developers full visibility into agent behavior and the ability to evaluate agentic systems in production. Amazon also contributes Cedar, an open source authorization language that enforces deterministic, verifiable boundaries on what AI agents are permitted to do, giving customers fine-grained, analyzable access controls to help ensure only authorized actions reach enterprise resources.

Capital One open sourced VulnHunter for agentic AI code security.

Cloudflare is offering its Vulnerability Discovery Harness as an open source skill to add security to agent systems.

Microsoft AI Red Team has open sourced several tools and harnesses. PyRIT – Python Risk Identification toolkit enables AI red teamers to run automated red teaming, with built in memory, supporting common targets, as well as custom endpoints.

RAMPART turns red-team findings and real-world incidents into repeatable tests that run as software changes. Clarity helps teams question design assumptions and identify potential failures before code is written.

Microsoft has also open sourced Assert, which converts natural language requirements and expected AI safety and security behaviors into executable evaluations.

Atlas is Wiz’s autonomous vulnerability research engine that orchestrates specialized AI agents to discover and validate security flaws across code and open source packages.

Visa has also joined the Open Secure AI Alliance, contributing its open sourced Visa Vulnerability Agentic Harness to help teams identify issues, support remediation and validation, quickly and safely.

Models — Intelligence Built for AI Safety and Defense

Not every security or safety task calls for a general-purpose model. Specialized security and safety models are purpose-built for defense: trained to understand code, locate vulnerabilities and reason about threats at scale. They can work to support agentic workflows as systems of models, with both open and closed models working together to get the job done efficiently.

Cisco DefenseClaw is an open source agentic governance layer that sits on top of NVIDIA OpenShell to provide robust, automated security at the runtime level when scaling agentic workforces. Cisco has also released two of its Antares security small language models to help pinpoint where known vulnerabilities exist within a codebase; and Project CodeGuard to embed secure-by-default practices directly into AI coding workflows.

阅读原文
AI Leaders Propose SAFE Guidelines for Cybersecurity Transparency · AI Daily