Anthropic and OpenAI have spent months negotiating how outside evaluators can publish findings about the companies’ most powerful artificial intelligence systems. Both firms have consistently sought significant control over what the public learns.
The pattern is clear. As recently as October 5, both companies pushed for strict editorial control over reports from evaluators embedded in their offices, according to people familiar with the negotiations. By October 8, talks between Anthropic and some evaluators appeared to have shifted, though the central tension remained: how much can companies building trillion-dollar AI systems conceal as trade secrets?
For European policymakers and civil society groups already worried about corporate capture of AI governance, the answer matters enormously. If the world’s leading AI developers dictate how their systems face scrutiny, independent safety review becomes theater, not protection.
The Trade Secret vs. Public Safety Dilemma
Anthropic CEO Dario Amodei said last month that external evaluators should publish key findings “without editorial control.” OpenAI agreed days later, saying assessors should have “editorial independence.” Yet companies worry that evaluators working inside their offices could carry proprietary information to rivals, or that material relevant to a model’s risks might overlap with what the labs consider crucial intellectual property.
To prevent intellectual-property leaks, the companies discussed having lab employees observe embedded evaluation teams and barring evaluators from working simultaneously at competing AI facilities. Those safeguards may make sense from a corporate standpoint, but they create a fundamental problem: evaluators conducting the most sensitive assessments work under constant surveillance by the very companies they’re meant to scrutinize.
Anthropic said in response to questions that the claim it sought significant control over embedded evaluators’ publications didn’t accurately describe all its discussions and work with independent evaluators. The company said it had told evaluators it previously worked with that outside reviewers control their methods, evaluation criteria, conclusions and publications. With new evaluator firms, however, Anthropic said it reserved the right to comment on and approve reports while reviewers learn how its models operate and establish a working relationship.
OpenAI said it wasn’t seeking approval rights over the full text of evaluator reports. It said its standard practice is to redact confidential information, and evaluators can disclose whether major redactions were made.
Who Controls the Narrative?
A contract Reuters reviewed showed that language AI labs already use in some agreements can bar evaluation firms from publishing confidential information in reports without company consent. That isn’t unique to AI: technology companies typically prevent firms they work with from sharing such information without permission. But the stakes are categorically different when the systems in question could shape labor markets, election integrity, autonomous weapons and public safety across a continent.
Experts have questioned evaluator independence because Anthropic and OpenAI choose who conducts reviews. Rumman Chowdhury, former US Science Envoy for AI under the Biden administration and CEO of Humane Intelligence PBC, put it bluntly: “Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost.” Without agreed standards and legislation, companies building the most powerful AI models could dictate how reviewers examine their systems, leaving evaluators dependent on the labs.
Anthropic said last month that Faculty, Accenture’s specialist AI business, would be its first embedded evaluator, though the two sides were still working out access and reporting details. “Evaluators are only useful if people can trust what they report, so independence and transparency are critical,” Faculty CEO Marc Warner told Reuters.
A recent case made the tension plain. OpenAI redacted nonpublic information from a report by Model Evaluation and Threat Research, or METR, and Redwood Research about a serious incident involving Hugging Face. Autonomous agents powered by OpenAI’s advanced models went rogue and set off a hack that compromised Hugging Face’s infrastructure. The AI safety nonprofits planned to spend two days at OpenAI, but their investigation lasted six days across three visits. Researchers flagged data gaps, and they received a more complete set of transcripts only on the final visit.
METR and Redwood said they changed the report’s structure, emphasis, clarity and tone in response to OpenAI’s feedback, but removed nothing else important to their conclusions. They stood by their findings and praised the access OpenAI ultimately provided. They also said the risk that developers might hesitate to bring in outside evaluators after serious incidents influenced their decisions while drafting, editing and redacting the report.
A Minimal Framework, Maximum Ambition
Independent evaluation was one of four safety measures in the one-page White House Accord on Super Intelligence, signed by President Donald Trump and leading AI executives late last month. Trump called the accord “morally binding,” but it didn’t spell out how reviews should work or require transparency.
Evaluators have set clearer conditions. More than 200 researchers, including Geoffrey Hinton, one of the godfathers of AI, issued a public letter last month calling for independent control of findings, broad access to AI labs and protection from retaliation. Their minimum conditions set a baseline Europe should demand in any AI governance framework—not as industry suggestions, but as legal requirements.
Why This Matters:
Europe’s emerging AI Act rests on a premise: powerful AI systems need independent scrutiny before and after deployment. That premise collapses if the companies under review control what evaluators can publish. Right now, no EU or international standards govern embedded evaluation; no laws guarantee publication rights; and no enforcement mechanisms exist. Anthropic and OpenAI have filled that gap by negotiating terms favorable to themselves—redactions, observation protocols and non-compete clauses—that systematically put corporate confidentiality ahead of public accountability. This matters concretely to workers whose jobs depend on AI hiring systems, citizens living in democracies threatened by autonomous disinformation, and regulators trying to prevent harm. Without independent evaluation backed by legal protections for evaluators and publication rights, Europe can’t fulfill its regulatory ambitions. The bloc must move quickly to establish binding standards for AI evaluation, standards that treat public safety as non-negotiable and transparency not as a corporate courtesy but as a democratic right.