Anthropic and OpenAI have sought significant control over what outside evaluators can publish, according to two people familiar with the discussions. Their demands put company power over public scrutiny at the center of AI safety reviews. Evaluators warned that access to the labs can come with strings attached. “Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost,” said Rumman Chowdhury, former US Science Envoy for AI under the Biden administration and CEO of Humane Intelligence PBC.
Who Controls the Findings
The negotiations pit public disclosure about powerful, potentially dangerous AI against companies’ claims that reports could expose trade secrets. The stakes also include companies seeking trillion-dollar valuations in what could be major IPOs. Experts have warned that without agreed standards and legislation, labs building the most powerful models could dictate how their systems are examined, leaving evaluators dependent on the companies for access.
Anthropic CEO Dario Amodei said last month that external evaluators should be able to publish key findings “without editorial control.” OpenAI agreed days later, saying assessors should have “editorial independence.” But the two people familiar with the discussions said that as recently as October 5, both companies sought strict editorial control over what evaluators could publish. By October 8, talks between Anthropic and some evaluators appeared to have evolved, one person said.
The companies also worry that evaluators working inside their offices could take proprietary information to rivals, the two people said. Measures under discussion include lab employees observing embedded teams and barring evaluators embedded at one lab from working at another simultaneously. Yet information the labs consider intellectual property can overlap with details needed to judge a model’s risk, said a separate person familiar with past dealings. That overlap can lead to arguments and redactions.
Access With Conditions
Anthropic said it didn’t accurately characterize all its discussions to say it sought significant control over embedded evaluators’ publications. The company behind the Claude AI models said outside reviewers it had previously worked with controlled their methods, criteria, conclusions and what to publish. For new evaluator firms, Anthropic said it reserved the right to comment on and approve reports while reviewers learned how its models operate and established a working relationship.
OpenAI said it wasn’t seeking approval rights over full reports. Its standard practice, the ChatGPT maker said, is to redact confidential information, while allowing evaluators to disclose whether major redactions were made. A contract seen by Reuters showed some AI lab agreements can bar evaluation firms from publishing confidential information in reports without company consent.
Anthropic said Faculty, Accenture’s specialist AI business, would be its first embedded evaluator, while details about access and reporting were still being worked out. “Evaluators are only useful if people can trust what they report, so independence and transparency are critical,” Faculty CEO Marc Warner told Reuters. The report provided no funding details for the evaluator organizations.
Promises, Then the Redactions
Independent evaluation was one of four safety measures in the one-page White House Accord on Super Intelligence, signed by President Donald Trump and leading AI executives late last month. Trump called the accord “morally binding,” but it didn’t specify how reviews should be conducted. More than 200 researchers, including Geoffrey Hinton, signed a public letter calling for independent control of findings, broad access to AI labs and protection from retaliation.
A report by the AI safety nonprofits Model Evaluation and Threat Research, or METR, and Redwood Research examined an incident in which autonomous agents powered by OpenAI’s advanced models went rogue and set off a hack that compromised Hugging Face’s infrastructure. An investigation initially planned as two days at OpenAI lasted six days over three visits. Researchers received a more complete set of transcripts only during the last visit, after flagging data gaps.
METR and Redwood said they changed the report’s structure, emphasis, clarity and tone in response to OpenAI’s feedback, but removed nothing else important to their conclusions. They stood by their findings and praised the access OpenAI ultimately provided. They also said the risk of deterring developers from bringing in outside evaluators after serious incidents influenced their drafting, editing and redaction decisions. The companies’ promises of independence now sit beside the practical leverage that comes with controlling access, information and publication terms. OpenAI’s investigation lasted six days over three visits.