Google Pushes AI Into Schools as Children Bear Risks

Google faces bipartisan Senate questions about bringing artificial intelligence into U.S. classrooms, where students rely on its tools for schoolwork and personal problems. Sens. Richard Blumenthal (D-Conn.) and Josh Hawley (R-Mo.) sent a letter Wednesday to Google CEO Sundar Pichai, seeking information about risks to children and the company’s decision to make AI integration the default on educational devices.

Who Pays

The senators wrote that they had “grave concern” that Google’s “aggressive push of AI products to children in schools is threatening the academic success and overall wellbeing of an entire generation.” They asked what Google knows about potential harms and whether its classroom tools had “encouraged a minor to engage in harmful behavior, emotional dependence, or failed to detect signs of suicidal or self-harm ideation.”

The letter cited a Maryland teenager who was hospitalized after developing an intense dependence on Google’s AI. The lawmakers asked Google to describe safeguards against inappropriate emotional exchanges between chatbots and children.

A Sept. 30 investigation by The Wall Street Journal reported that children increasingly relied on AI to complete assignments, answer basic questions and navigate personal problems. The newspaper interviewed more than 60 teachers, school administrators, parents, students and clinicians.

Google researchers had warned about possible effects on children’s development and emotional well-being, according to the Journal’s review of more than a dozen company research papers and interviews with current and former employees. The Journal’s reporting also raised questions about the company’s promotion of the technology despite warnings from its own researchers.

Behavioral scientist Dr. Abbie Maroño told The Post that children need chances to work through uncertainty, make mistakes and learn to solve problems with support. “If AI repeatedly supplies the answer before they have tried, it can remove opportunities to develop those skills,” she said.

Maroño also warned that reassuring chatbot responses can create a misleading sense of trust. “A response can feel caring without being accurate or appropriate for that child,” she said. She called for independent testing of how systems respond to distress, repeated reassurance-seeking and a child treating a chatbot as their only source of support.

The Company’s Response

Google said Gemini has special safeguards intended to prevent children from developing emotional attachments to the technology or receiving responses that reinforce potentially harmful beliefs. In an April 7 policy statement, the company described “protections designed to prevent Gemini from acting like a companion,” including avoiding “language that simulates intimacy or expresses needs.”

Google said it trained Gemini “not to agree with or reinforce false beliefs” and to direct users toward objective information and real-world support. The company also said it was introducing a simplified “one-touch” interface connecting people experiencing suicidal thoughts or other mental health crises with professional support by calls, texts or online chats.

Google.org, the company’s philanthropic arm, announced a $30 million commitment over three years to support crisis hotlines worldwide. Google acknowledged that AI presents new mental health challenges and said Gemini “is not a substitute for professional clinical care, therapy, or crisis support for those who need it.”

The Senate Inquiry

The senators also requested an accounting of Google’s promotional efforts to expand AI use in schools, including how much the company spent. Their letter said, “Google is forcing their AI products into classrooms across America, and children are paying the price.” The article provides no figure for promotional spending.

The senators gave Google until Nov. 2 to answer. The company hasn't responded to a request for comment. The letter seeks information about the risks, safeguards and promotion of products Google has pushed into classrooms; the company’s public statements describe its protections and crisis-support plans, while the senators’ questions remain unanswered in the account. There’s no spending figure in the article, and Google hasn’t answered the senators by the deadline stated here. It's a dispute over classroom AI, and children are paying the price.