Google's Classroom AI Push Faces Senate Scrutiny

Google made AI integration the default on educational devices, and now bipartisan senators are asking what the company knows about risks to students’ academic performance and mental health. The inquiry has turned a corporate technology decision in U.S. classrooms into a dispute over who sets the terms of children’s learning.

Who Sets the Classroom Terms

Sens. Richard Blumenthal (D-Conn.) and Josh Hawley (R-Mo.) sent Google CEO Sundar Pichai a letter Wednesday, seeking information about potential harms from the company’s AI products for children. They asked why Google made AI integration the default on educational devices and how much the company knows about risks to students.

The lawmakers said they had “grave concern” that Google’s “aggressive push of AI products to children in schools is threatening the academic success and overall wellbeing of an entire generation.” They also asked whether classroom tools had “encouraged a minor to engage in harmful behavior, emotional dependence, or failed to detect signs of suicidal or self-harm ideation.”

The letter seeks details on safeguards against inappropriate emotional exchanges between chatbots and children. The senators cited a Maryland teenager who was hospitalized after developing an intense dependence on Google’s AI. They also requested an accounting of Google’s promotional efforts to expand AI use in schools, including how much the company spent.

“Google is forcing their AI products into classrooms across America, and children are paying the price,” Blumenthal and Hawley wrote. Google must answer by Nov. 2.

Evidence and Concerns

The inquiry came after a Sept. 30 investigation by The Wall Street Journal into Google AI tools and students’ academic and emotional dependence, along with questions about the company’s promotion of the technology despite warnings from its own researchers. The newspaper interviewed more than 60 teachers, school administrators, parents, students and clinicians.

Those interviewed described children increasingly relying on AI to finish assignments, answer basic questions and handle personal problems. The Journal reviewed more than a dozen company research papers and spoke with current and former employees. Its review found that Google researchers warned the technology could undermine children’s cognitive development and emotional well-being.

Children need room to work through uncertainty. They need to make mistakes, too. Behavioral scientist Dr. Abbie Maroño told The Post that children need opportunities to tackle uncertainty, make mistakes and learn they can solve problems with support. “If AI repeatedly supplies the answer before they have tried, it can remove opportunities to develop those skills,” she said.

Maroño warned that a chatbot’s reassuring response can foster misplaced trust. “A response can feel caring without being accurate or appropriate for that child,” she said. She called for independent testing of how systems respond to distress, repeated reassurance-seeking and a child treating a chatbot as their only source of support.

Google’s Safeguards and Commitments

Google said its Gemini chatbot has special safeguards meant to keep children from developing emotional attachments to the technology or receiving responses that reinforce potentially harmful beliefs. In an April 7 policy statement, the company described “protections designed to prevent Gemini from acting like a companion.” It said safeguards include avoiding “language that simulates intimacy or expresses needs.”

Google said it trained Gemini “not to agree with or reinforce false beliefs” and to direct users toward objective information and real-world support. It also said it was introducing a simplified “one-touch” interface to connect people experiencing suicidal thoughts or other mental health crises with professional support through calls, texts or online chats.

Google.org, the company’s philanthropic arm, announced a $30 million commitment over three years to support crisis hotlines worldwide. Google acknowledged AI brings new mental health challenges and cautioned that Gemini “is not a substitute for professional clinical care, therapy, or crisis support for those who need it.” The senators want Google to account for its classroom expansion, the safeguards it describes and the promotional spending they’ve asked the company to disclose.