President Donald Trump praised AI executives for their “tremendous self-policing” and signaled he intends to let the companies operate on their own rather than impede an industry driving the economy and stock market. That hands-off stance has opened space for independent evaluators, including Model Evaluation and Threat Research (METR), Apollo Research and Transluce, to take on a role the government hasn’t filled.
Mostly nonprofits, these groups assess model capabilities and risks, and flag cases where the technology behaves badly. Anthropic and OpenAI now want their support. With no federal push for regulation, evaluators have gained real importance, and private capital is following.
The Private-Sector Response
Money is moving fast. Rayan Krishnan, CEO of Vals AI, said the for-profit startup, which builds benchmarks for measuring AI models on industry-specific tasks, grew from eight employees to roughly 30 this year and announced a $40 million funding round in August. METR said in August that it had received commitments of around $71 million over the previous six months. Its total contributions in 2024 came to $13.6 million, according to its most recent IRS filing.
The field is maturing quickly, said Andrew Freedman, CEO of policy nonprofit Fathom. “I’ve worked in politics and policy for the last 20 years of my life, and I’ve never seen an issue move so fast on so many different political spectrums,” he said, adding that he expects an “influx of capital.”
Last month, Anthropic CEO Dario Amodei pledged to embed independent evaluators at his company, and OpenAI CEO Sam Altman endorsed the move. Amodei said evaluators would get desks, access badges, company laptops and permissions “mostly comparable” with those of internal risk assessment teams. Contracts would grant them “the right to publish key findings,” while Anthropic kept “the narrow ability” to redact security-sensitive or confidential information. Anthropic said it would fund Accenture’s contributions directly.
A Voluntary Accord, and Its Critics
The industry’s closest thing to standards is a “morally binding” one-page agreement Trump presented at a White House luncheon in late September. It says “every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public,” and encourages signatories to work with an “independent external auditor or evaluator.” Top executives at Anthropic, Google, Meta, OpenAI, SpaceX and Nvidia signed it.
Suresh Venkatasubramanian, a computer science professor at Brown University, called it “a performance of an attempt to show action when in fact no action actually happened.” The companies promised things “they should have been doing already,” he said. He also raised the funding question: “Who is paying for these companies to do their work? You need an ecosystem, you need a viable business model for this.”
Critics say the leading labs are writing the rules, comparing the arrangement to asking the biggest banks to protect against a financial crisis. Kevin Werbach of the Wharton Accountable AI Lab said the ecosystem “isn’t robust enough right now.” METR employs fewer than 50 full-time staffers; the leading labs have raised tens of billions of dollars and employ thousands.
Friction Inside OpenAI
The arrangement already faces strain. Last week, OpenAI fired three employees for “violating our policies on accessing and handling sensitive company information,” a spokesperson said. Two of them, Mikita Balesni and Tomek Korbak, said they believed the company dismissed them because of how they communicated with third-party evaluators. Balesni wrote on X that former colleagues “are afraid to speak.”
OpenAI disputed that account, saying it was “actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks.” An OpenAI spokesperson said the coming work builds on existing collaboration with METR and Redwood Research. Anthropic didn’t respond to a request for comment.
Licensing Enters the Debate
Introduced in June of last year, Fathom’s marketplace framework would have the government license Independent Verification Organizations (IVOs) to test whether AI companies meet safety criteria. Freedman said government oversight is needed so evaluators don’t become dependent on labs for revenue and feel encouraged to “start rubber-stamping stuff.”
IVOs are a key provision of the FRONTIER Act, introduced in July by Reps. Lori Trahan, D-Mass., and Jay Obernolte, R-Calif. Lawmakers in California, Connecticut and Virginia have taken steps toward IVOs, while Massachusetts is weighing independent safety evaluations more broadly. California Governor Gavin Newsom signed two bills: one establishing a “first-in-the-nation framework” and another creating a state registry for AI auditors. OpenAI endorsed them the day he signed. Chris Lehane, OpenAI’s global affairs chief, wrote that “we prefer independent technical assessments to be required at the federal level,” but that without federal action, “California can help establish the rules of the road.”
Freedman gave a blunt read on the companies’ motives. “I don’t think you need to trust that they’ve suddenly turned altruistic or that there’s anything but corporations acting like corporations,” he said.
Why This Matters:
The fight isn’t about whether AI needs scrutiny. Everyone from Trump to Amodei agrees it does. The question is who builds the machinery and who pays. A voluntary, market-driven evaluator ecosystem keeps Washington from imposing a heavy regulatory apparatus on an industry driving the economy and stock market. But the facts show clear limits: tiny nonprofits assessing firms with tens of billions in funding, no settled standards for access or reporting, and the risk that evaluators depend on the companies they judge. State-by-state licensing, led by California, could leave businesses with a patchwork of rules. If private evaluation is to replace federal mandates, its credibility will depend on independence that, so far, hasn’t been secured.