AI Firms Automate Research as Oversight Lags

Leading AI companies are working to automate every stage of development, from coming up with research ideas to running experiments and interpreting results, according to a report by Gladstone AI. Most of the half dozen researchers at top AI companies who spoke with the report’s authors believe recursive self-improvement, or RSI, could begin soon—and perhaps sooner if there’s no U.S. government intervention or slowdown. Gladstone CEO Jeremie Harris said the report describes a research process increasingly shaped by companies’ computing resources, while lawmakers have “essentially zero visibility into” how companies identify promising ideas.

The process under development

Shared exclusively with TIME before publication on Wednesday, the Gladstone report details how RSI might work and how companies are progressing toward it. Anthropic and OpenAI have warned for months that AI may be nearing the point where it can improve itself. RSI could bring new capabilities faster, but developers could also lose control of AI systems entirely, the report says.

The report describes pretraining as a search for ways to get more intelligence from a model while using less computing power, known in the industry as “compute.” Researchers test portfolios of ideas by assigning a small amount of compute to train a series of models. Ideas that reliably improve performance as models scale then get another training round with more compute.

That sequence ties research choices to access to computing resources. Limited compute restricts how many promising ideas companies can test at larger scales, while companies with more compute can try more ideas and may reach RSI first, according to the report. More compute may also help companies spot ideas that perform better as computing power increases, even when those ideas initially seem to fail at smaller scales.

Who sees the decisions

AI-generated ideas don’t automatically produce useful research. The report says the number of poor ideas matters because sorting good ideas from bad ones can take so much time or compute that companies may prefer human researchers with sound intuition. Automating parts of the pipeline therefore means not only generating ideas, but testing them and interpreting results.

Jeremie Harris said legislators currently have “essentially zero visibility into” how companies identify such ideas and whether AI is helping find them. The report puts those research choices inside companies, where access to compute and the ability to filter results affect which ideas advance to larger tests.

Regulation and control

Gladstone executives said they want lawmakers to better understand the process and how it might be regulated. Jeremie Harris called proposed bills to ban superintelligence “really interesting attempts to grapple with this problem,” then asked: “what is the process that needs to be regulated?”

Edouard Harris, Gladstone’s chief technology officer, said regulation is difficult because RSI develops gradually. “This is actually one of the reasons why we went to pains to describe it in so much detail,” he said, “so that it is possible to point to a part of this process and say: a human needs to be there.”

The report also records researchers’ concern. “There’s a fair amount of [disaster] prepping and bio-bunker purchasing going on,” Edouard Harris said. Jeremie Harris said the report aimed to define RSI well enough for “a sane conversation about it, especially in Washington.”