AI firms race toward self-improvement with little oversight

The world’s leading artificial intelligence companies are quietly automating the entire process of building better AI systems, from generating research ideas to running experiments and analyzing results. A new investigation raises urgent questions about democratic accountability and public safety.

A report by Gladstone AI, shared exclusively with TIME before publication Wednesday, documents how major firms including Anthropic and OpenAI are moving toward recursive self-improvement, or RSI—a capability that would let AI systems improve themselves without human intervention. Researchers inside these companies estimate RSI could begin within the next 3 to 12 months, and possibly sooner if there’s no U.S. government intervention or slowdown.

The stakes couldn’t be higher. RSI could accelerate AI capabilities dramatically. It could also leave developers unable to control AI systems. Yet policymakers, the public, and even many people outside the AI industry can barely see the process.

The Automation Problem

Currently, AI development follows a structured pipeline. Companies spend enormous resources testing different research ideas: they allocate computing power to train models, then select the approaches that work best. Next, they invest more resources in the most promising ideas and scale up their experiments. It’s a human-guided process—slow, expensive, and constrained by limited computing resources.

But companies are automating the entire cycle. AI itself is beginning to generate research ideas, run experiments, and interpret results. The problem isn’t hard to see: companies with the most computing power will likely reach RSI first, widening the competitive advantage of firms that already dominate. The barrier to entry keeps climbing. Startups and smaller research institutions fall further behind.

Jeremie Harris, CEO of Gladstone AI, told TIME that legislators currently have “essentially zero visibility into” how companies identify promising research ideas or whether AI helps them do so. This isn’t a minor transparency gap—it’s a governance failure. The most consequential technology in decades is being developed in shadows, guided by market competition rather than public interest or democratic oversight.

A Gradual Shift With No Checkpoints

One of the report’s most troubling findings is that RSI doesn’t announce itself in a single dramatic moment. Instead, it develops gradually and incrementally across the AI development pipeline. That makes it nearly impossible for regulators to identify the precise point when intervention is needed.

Edouard Harris, Gladstone’s chief technology officer, put the difficulty bluntly: “This is actually one of the reasons why we went to pains to describe it in so much detail, so that it is possible to point to a part of this process and say: a human needs to be there.”

The irony is bitter. Gladstone’s researchers are essentially asking regulators to design safeguards for a process that AI companies are deliberately obscuring. Harris added that there’s “a fair amount of [disaster] prepping and bio-bunker purchasing going on” among AI researchers—a stark acknowledgment that even the people building these systems are preparing for worst-case scenarios.

The Regulation Gap

Both Anthropic and OpenAI have publicly warned for months that AI systems may be approaching the threshold for self-improvement. Yet proposed legislation to address this remains vague. Jeremie Harris said bills to ban superintelligence are “really interesting attempts to grapple with this problem,” but the real question lawmakers haven’t answered is: “what is the process that needs to be regulated?”

Without clear answers, regulation becomes reactive rather than preventive. Companies keep building toward RSI while the government remains, as Harris said, essentially blind to how it’s happening. That’s not how democratic societies should manage transformative technologies. Public institutions should set the terms. Markets should operate within those boundaries.

Instead, private companies race ahead while legislators scramble to understand what they’re racing toward.

Why This Matters:

Automating AI development represents a fundamental shift in power and risk. If a handful of well-capitalized companies reach RSI first, they’ll control a technology that could reshape society—its economy, its security, and its democratic institutions. The public will have had no say in how that happened or what safeguards companies built in. Gladstone’s report warns that we’re running out of time to demand public oversight before these systems escape meaningful human control. Researchers are right to demand that lawmakers understand the process in detail. Without that understanding, and without regulatory action grounded in public interest rather than corporate competition, we’re betting our future on profit-driven companies choosing safety over speed.