New York City lawmakers left Monday’s hearing with few answers from major A.I. companies about catastrophic risks, insurance, or safeguards for models that fail safety reviews. The companies didn’t quantify the chance of a cataclysmic event, and they didn’t say whether they could stop a model after it failed an internal safety test or external review.
Questions Without Answers
At Monday’s City Council hearing, City Council speaker Julie Menin asked company representatives how they were quantifying “the risk imposed by A.I. in the worst case catastrophic scenario.” The hearing focused on risks tied to the companies’ technology and the safeguards they planned to provide.
Morgan Dwyer, who runs OpenAI’s policy development and operations, answered: “I don't know.” She added, “I also don't think it matters whether it's 1 percent or 10 percent or a 20 percent chance that something catastrophic will go wrong.”
Menin rejected that response. “None of these levels is remotely acceptable,” she said. “I'm sorry, but to say you don't know and it doesn't matter is flippant at best,” Menin replied.
The unanswered risk estimate took center stage. Dwyer gave no percentage for the likelihood of a catastrophic event. The article does not report that the companies offered another estimate during the hearing.
Safeguards and Liability
Lawmakers also asked whether the companies had insurance for catastrophic events. The companies didn’t say whether they had such insurance. The hearing offered few assurances about how the industry would guard against catastrophic scenarios.
The questions went beyond measuring risk. Company representatives also didn’t explain whether they could stop an A.I. model that failed an internal safety test or an external review. The hearing account gives no company response describing a mechanism to halt such a model.
Dwyer said OpenAI shouldn’t train models that can’t be safely kept under human control. Her statement addressed what OpenAI should do during training; it didn’t specify a way to stop a model after a failed test or review. The article reports no further details from her on that question.
A Hearing With Few Assurances
The City Council hearing put company representatives before lawmakers asking about the risks their technology could pose. Menin sought a quantified assessment of the worst-case scenario, while Dwyer said she didn’t know the probability and didn’t think the precise percentage mattered.
The source reports no profit figures, insurance terms, worker impacts, union activity, or organized resistance related to the companies. It also reports no regulatory proposal or company commitment resulting from the hearing. Its account ends with lawmakers still lacking clarity about how the industry would address looming threats.
Lawmakers asked about risk, insurance, human control, and stopping models that fail safety checks. The companies offered little clarity on those points. The City Council’s questioning yielded no detailed safeguards or assurances against catastrophic scenarios.