New York City lawmakers questioned representatives of major A.I. companies at a City Council hearing on Monday, but left with little clarity about how the companies planned to address looming threats. The companies didn’t quantify the risk of a cataclysmic event, say whether they had insurance for one or explain whether they could stop a model that failed an internal safety test or external review. The hearing produced few answers or assurances about safeguards against catastrophic scenarios.
What lawmakers asked
City Council speaker Julie Menin asked company representatives how they were quantifying “the risk imposed by A.I. in the worst case catastrophic scenario.” She wanted a measure of the risk tied to the worst-case outcome. According to the account, the companies didn’t provide a quantified estimate at the hearing.
Morgan Dwyer, who runs OpenAI's policy development and operations, answered, “I don't know.” She added, “I also don't think it matters whether it's 1 percent or 10 percent or a 20 percent chance that something catastrophic will go wrong.” Her response gave no probability for a catastrophic event.
Menin replied, “None of these levels is remotely acceptable.” She then told Dwyer, “I'm sorry, but to say you don't know and it doesn't matter is flippant at best.” The exchange laid bare their disagreement over risk estimates: Menin asked how the risk was being measured, while Dwyer said she didn’t know and argued that the precise percentage didn’t matter.
Safeguards left unclear
Dwyer said OpenAI shouldn’t train models that can’t be safely kept under human control. The hearing account doesn’t describe a specific procedure for stopping a model after it fails an internal safety test or external review. Representatives also didn’t explain whether their companies could halt such a model.
The companies didn’t say whether they had insurance for cataclysmic events. Nor did they quantify the risk of such an event. Those questions remained unanswered as lawmakers sought information about the industry's plans to address looming threats.
Few assurances from the hearing
The City Council hearing brought representatives of major A.I. companies before lawmakers, but the exchange yielded little clarity about the companies' safeguards. The account identifies no specific insurance coverage, risk estimate or shutdown capability offered by the representatives. It reports Dwyer's position on human control and her response to Menin's question, but gives no further assurance about how that position would be put into practice.
The central exchange ended without a quantified answer to Menin's question. Dwyer said she didn’t know the risk figure and didn’t think the difference between a 1 percent, 10 percent or 20 percent chance mattered. Menin rejected that response. The hearing yielded few answers or assurances about how the industry would guard against catastrophic scenarios.