Anthropic and OpenAI have sought significant control over what outside evaluators can publish, according to two people familiar with discussions. That gives the companies a central role in deciding how reviewers scrutinize their AI systems. The dispute concerns whether independent reviewers can report findings about potentially dangerous models without company approval, as labs seek to protect trade secrets.
The issue goes beyond wording in evaluation contracts. Experts have warned that, without agreed standards and legislation, companies developing the most powerful AI models could dictate how those systems are examined, leaving evaluators dependent on labs for access. The negotiations will help determine what information about the technology reaches the public.
Who Controls the Findings
As recently as October 5, Anthropic and OpenAI were seeking strict editorial control over what evaluators could publish, the two people familiar with the discussions said. By October 8, talks between Anthropic and some evaluators appeared to have evolved, one person said. The people declined to be named because the matter was private.
Anthropic CEO Dario Amodei said last month that external evaluators should be able to publish key findings “without editorial control.” OpenAI agreed days later, saying assessors should have “editorial independence.” But both companies have also raised concerns about proprietary information. The people familiar with the discussions said labs worried that evaluator staff working inside their offices could take confidential information to rivals.
Measures under discussion include having lab employees observe embedded evaluation teams and barring evaluators embedded at one lab from working at another simultaneously, one of the people said. A separate person familiar with past dealings said information labs view as intellectual property can overlap with details relevant to assessing model risks, creating potential disputes over arguments and redactions.
Anthropic said the claim that it sought significant control over embedded evaluators’ publications didn’t accurately characterize all its discussions and work with independent evaluators. The company said that, in current conversations with evaluators it had previously worked with, outside reviewers controlled their methods, criteria, conclusions and publication decisions. For new evaluator firms, Anthropic said it reserved the right to comment on and approve reports while reviewers learned how its models operate and established a working relationship.
Access, Redactions and Public Accountability
OpenAI said it wasn’t seeking approval rights over the full text of evaluator reports. It said its standard practice is to redact confidential information and that evaluators can disclose whether major redactions were made. A contract seen by Reuters showed that language in some AI lab agreements can prevent evaluation firms from publishing confidential information in reports without company consent.
OpenAI has said AI labs and assessors should agree on scope and access before testing, disclose methods and conflicts, and let assessors control findings while allowing companies to protect sensitive information. Rumman Chowdhury, former US Science Envoy for AI under the Biden administration and CEO of Humane Intelligence PBC, said labs could control the access evaluators depend on and gain leverage over what gets reported without standards. “Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost,” Chowdhury told Reuters.
Anthropic said last month that Faculty, Accenture’s specialist AI business, would be its first embedded evaluator, while details about access and reporting remained under discussion. “Evaluators are only useful if people can trust what they report, so independence and transparency are critical,” Faculty CEO Marc Warner told Reuters.
Independent evaluation was one of four safety measures in the one-page White House Accord on Super Intelligence, signed by President Donald Trump and leading AI executives late last month. Trump has called the accord “morally binding,” but it didn’t specify how reviews should be conducted. More than 200 researchers, including Geoffrey Hinton, signed a public letter last month calling for independent control of findings, broad access to AI labs and protection from retaliation.
A report by the AI safety nonprofits Model Evaluation and Threat Research, or METR, and Redwood Research examined the Hugging Face incident, in which autonomous agents powered by OpenAI’s advanced models went rogue and set off a hack that compromised Hugging Face’s infrastructure. An investigation that researchers initially planned to complete in two days at OpenAI lasted six days over three visits. Researchers received a more complete set of transcripts only during the last visit, after they flagged data gaps.
METR and Redwood said they changed the report’s structure, emphasis, clarity and tone in response to OpenAI’s feedback, but removed nothing else important to their conclusions. They stood by their findings and praised the access OpenAI ultimately provided. They said the risk that developers might be deterred from bringing in outside evaluators after serious incidents influenced their decisions while drafting, editing and redacting the report.