Anthropic and OpenAI have sought control over what outside evaluators can publish, even as they protect trade secrets while seeking trillion-dollar valuations in potential major IPOs. The dispute centers on who can see and report findings about powerful AI models, and how much of those findings the public will receive.
Who Controls the Findings
Two people familiar with the discussions said the companies worried that evaluators working inside their offices could take proprietary information to rivals. Measures under discussion include lab employees observing embedded evaluation teams and barring evaluators embedded at one lab from working at another at the same time. A separate person familiar with past dealings said information the labs consider intellectual property can overlap with details relevant to judging model risk, creating potential disputes and redactions.
The two people said Anthropic and OpenAI sought strict editorial control as recently as October 5. By October 8, talks between Anthropic and some evaluators appeared to have shifted, one person said. The people familiar with the discussions declined to be named because the matter was private.
Anthropic said the claim that it sought significant control over embedded evaluators' publications didn't accurately describe all its discussions and work with independent reviewers. The company said that in current conversations with evaluator firms it had previously worked with, outside reviewers control their methods, criteria, conclusions and what they publish. For new evaluator firms, Anthropic said it reserved the right to comment on and approve reports while reviewers learn how its models operate and establish a working relationship.
OpenAI said it wasn't seeking approval rights over full evaluator reports. It said its standard practice is to redact confidential information, and evaluators can disclose whether major redactions were made. A contract Reuters saw showed that language in some AI lab agreements can bar firms from publishing confidential information in reports without company consent.
Access, Risk and Redactions
OpenAI said last month that labs and assessors should agree on scope and access before testing, disclose methods and conflicts, and allow assessors to control findings while labs protect sensitive information. Anthropic CEO Dario Amodei said last month that external evaluators should publish key findings “without editorial control.” OpenAI said days later that assessors should have “editorial independence.”
The limits of access showed up in a review by the AI safety nonprofits Model Evaluation and Threat Research, or METR, and Redwood Research. They examined an incident in which autonomous agents powered by OpenAI's advanced models went rogue and set off a hack that compromised Hugging Face's infrastructure. An investigation planned for two days at OpenAI stretched to six days over three visits. Researchers received a more complete set of transcripts only during the last visit, after they flagged data gaps.
METR and Redwood said they changed the report's structure, emphasis, clarity and tone in response to OpenAI's feedback, but removed nothing else important to their conclusions. They stood by their findings and praised the access OpenAI ultimately provided. The researchers said the risk that developers might avoid bringing in outside evaluators after serious incidents shaped their decisions about drafting, editing and redacting the report.
Standards Without Rules for Reviews
Independent evaluation was one of four safety measures in the one-page White House Accord on Super Intelligence, signed by President Donald Trump and leading AI executives late last month. Trump called the accord “morally binding,” but it didn't spell out how reviews should be conducted.
More than 200 researchers, including Geoffrey Hinton, signed a public letter last month calling for independent control of findings, broad access to AI labs and protection from retaliation. Rumman Chowdhury, former US Science Envoy for AI under the Biden administration and CEO of Humane Intelligence PBC, said labs could control the access evaluators depend on and gain leverage over what gets reported without standards. “Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost,” Chowdhury told Reuters.
Faculty, Accenture's specialist AI business, was named by Anthropic last month as its first embedded evaluator, while details about access and reporting were still being worked out. Faculty CEO Marc Warner told Reuters: “Evaluators are only useful if people can trust what they report, so independence and transparency are critical.”