AI Companies Race Toward Self-Improvement Without Clear Guardrails

AI companies are quietly automating the entire development pipeline with little government oversight, according to a Gladstone AI report shared exclusively with TIME before its Wednesday publication. Researchers at leading firms believe recursive self-improvement, or RSI—where AI systems improve themselves without human intervention—could arrive within the next 3 to 12 months, possibly sooner if regulators don't act. The stakes couldn't be higher.

This isn't theoretical speculation anymore. Half a dozen researchers inside leading AI companies told Gladstone's investigators they're actively working toward this capability. Anthropic and OpenAI have publicly warned for months that AI is nearing a threshold where it can iterate on itself. The report offers a detailed roadmap of how companies are progressing toward RSI, and reveals an uncomfortable truth: government visibility into the process is virtually nonexistent.

The Automation Pipeline

Understanding what's happening means understanding how modern AI development works. During pretraining, companies look for ways to draw more intelligence from their models while using less computing power—a metric the industry simply calls "compute." Researchers test dozens of promising ideas by assigning a small amount of compute to train a series of models. They select ideas that reliably improve performance as models scale for another round of training, using substantially more compute.

Here's where the problem emerges: limited computing resources restrict how many promising ideas companies can test at larger scales. AI might generate ideas that look promising on paper, but sorting the good ones from the bad takes precious time and computational resources. Companies with more compute can test more ideas, faster. That gives an advantage to firms able to automate testing most efficiently—and they may reach RSI first.

Edouard Harris, Gladstone's chief technology officer, offered a stark observation: "There's a fair amount of [disaster] prepping and bio-bunker purchasing going on" among researchers. That detail speaks volumes about how seriously people in the field view these risks.

The Regulatory Vacuum

Jeremie Harris, Gladstone's CEO and Edouard's brother, explained the report's purpose: "Our goal here was to define recursive self-improvement well enough that people can have a sane conversation about it, especially in Washington." He called proposed bills to ban superintelligence "really interesting attempts to grapple with this problem," but added a crucial qualification: "what is the process that needs to be regulated?"

That question gets to the heart of the governance challenge. Legislators have "essentially zero visibility into" how companies identify promising AI research ideas and whether AI systems already help find them, Harris said. The problem goes beyond ignorance. RSI is genuinely difficult to regulate because it develops gradually, almost imperceptibly, making the moment to intervene hard to pinpoint.

Edouard Harris addressed the difficulty directly: "This is actually one of the reasons why we went to pains to describe it in so much detail, so that it is possible to point to a part of this process and say: a human needs to be there." It's a remarkable admission: companies can't clearly identify where human oversight becomes essential, yet they're moving forward anyway.

Market Incentives and Government's Role

Market forces alone won't solve the classic problem created by these competitive dynamics. Every AI company has an incentive to automate faster and reach RSI first. The company that gets there doesn't necessarily win by acting most responsibly; it wins by moving fastest. This isn't speculation about corporate malfeasance. It's basic economics: when rewards favor speed over safety, speed typically wins until regulators change the rules.

Yet the regulatory response remains minimal. Government's lack of visibility creates a real gap between activity in laboratories and policymakers' understanding. Companies are asking for better-informed legislation, which is pragmatic, but the current moment offers a narrow window to put meaningful guardrails in place before RSI capabilities mature beyond easy control.

Why This Matters:

The fiscal and institutional implications are substantial. First, companies investing billions in AI development are making unilateral decisions about automation timelines without robust public accountability. Second, if self-improving AI systems emerge and companies lose effective control—a genuine possibility Anthropic and OpenAI have publicly flagged—the consequences could be catastrophic and impossible to contain. Third, today's competitive structure rewards speed over safety, creating perverse incentives that market forces alone won't correct. Fourth, legislators acknowledge having almost no visibility into how this critical technology is developing, meaning regulatory responses will necessarily lag behind capability development. Finally, researchers involved are reportedly taking disaster preparedness seriously enough to invest in bunkers, suggesting widespread concern inside the field about tail risks. This failure of normal institutional oversight to keep pace with technological development shows precisely where government's limited regulatory role becomes genuinely important.