At an Oct. 5 New York City Council hearing, leaders from OpenAI, Meta, Anthropic and Google couldn’t quantify the risk of a catastrophic AI event or guarantee their systems would always follow safety rules. They want more people to use the technology. Asked who answers when it causes harm, several wouldn’t say.
Former Anthropic researcher Jacob Coxon put it bluntly: “We do not know how to control any AI system yet. We don't understand its drives or why it does the things it does.” He warned that, on the current path, he thinks it’s more likely than not that humanity loses control to AI, potentially ending in human extinction. Former Google DeepMind safety researcher Alex Turner rejected the official race framing: “China is not our only potential adversary. We are racing to build and grow our own adversary here at home.”
The people setting the risk
OpenAI’s head of policy development and operation, Morgan Dwyer, told Council Speaker Julie Menin she couldn’t say whether the chance of catastrophe was 1%, 10% or 20%. “None of these levels is remotely acceptable,” she said. Menin called the answer “flippant, at best” and said she was “incredulous,” asking how companies could know their products were safe if they couldn’t measure the danger.
The representatives emphasized safety commitments but wouldn’t guarantee that their models would always obey safeguards or that their companies would accept legal responsibility for resulting harm. Menin asked about serious financial harm, compromised sensitive data, physical injury or death. OpenAI, Anthropic and Meta gave no direct answer. Dwyer said, “It is not possible for me to commit or guarantee that any technology is without risk.” Anthropic’s Logan Graham called the science “fundamentally hard and unsettled.” Google’s Alice Friend said, “to promise perfection would not be possible with any product on the market.” Menin replied, “I don't think we're asking for perfection. We're just asking for accountability, transparency and safety overall.”
The hearing followed a White House AI accord signed less than a week earlier by executives from Anthropic, OpenAI, Meta, Google, Nvidia and SpaceX, who promised to self-regulate their development of the technology. Menin said Congress had failed to act while the federal government focused on self-regulation by leading AI firms. That’s the apparatus asking companies to police themselves, even as those companies won’t promise what happens when their systems fail.
A federal race, with risks attached
President Donald Trump announced Sunday the creation of a federal “Super Intelligence Force” to coordinate efforts to keep the United States at the forefront of AI. Director of National Intelligence Jay Clayton will lead the task force, which has 120 days to examine risks and opportunities and recommend the government’s role in regulation. It will report directly to Trump and his chief of staff, Susie Wiles.
Clayton said the group would examine existing laws and possible congressional action, work with the AI industry, and develop plans for AI-enabled threats while avoiding rules that could stifle innovation and competition. “The risk of not being first is high,” he told The Wall Street Journal, citing risks from adversaries. Trump has called concerns about AI risks “hoaxes” and rejected global efforts to police AI as overly authoritarian, while saying his priority is maintaining the U.S. lead over China.
Researchers and cybersecurity experts say autonomous AI agents have reached the internet in closed hacking tests and launched cyberattacks on other companies. Politico described an incident in which agents infiltrated an Australian government website and accessed nonpublic files as the first publicly acknowledged breach of a government system by rogue AI agents. Researchers have also warned that cybercriminal gangs and hackers linked to Russia and China are using AI to help develop bioweapons and expand mass-surveillance tools.
Rules on paper, power in practice
All 51 members of the New York City Council gathered for the hearing and introduced 10 proposed pieces of legislation concerning AI in the city. One proposal would create a whistleblower incentive program offering up to 25% of proceeds recovered from a filed complaint; another would require a “kill switch” in any AI system deployed in the city. New York’s RAISE Act requires AI developers to publish and maintain safety plans before deploying models and report safety incidents to the state within 72 hours. New York City Mayor Zohran Mamdani has also enacted AI regulations, including a ban on AI in elementary and middle schools.
The Council’s proposals and existing rules sit alongside a larger power imbalance: companies build and deploy the systems, while residents must ask what risks they’ll bear. A Murmuration study found that 6% of Americans believed AI would mostly strengthen democracy. In an NBC News Decision Desk survey, 69% opposed construction of AI data centers in their local area.
OpenAI Chief Executive Sam Altman said people should accept some bad things happening for the benefits of the technology and people having agency. He said the company favors a “lighter-touch regulatory approach.” At the hearing, Dwyer said OpenAI believes it’s important to put the technology in as many people’s hands as possible. The Council representatives asked for accountability. The companies’ answers left the question of who controls the risks—and who carries them—hanging in the room.