Google Locks AI Into Kids' Search. Parents Say No.

Google confirmed last week that there will be no way for students to use Google Search without AI. No toggle. No opt-out. Parents were told the only way to dodge the machine was to type "-ai" at the end of every query. On Wednesday, families in seven cities walked out into the street to answer that.

In Chicago, dozens of protesters marched through cold, rainy weather to Google's downtown office, chanting "Hey Google, nice try, we don't want your Gemini" and "Hey, Google, leave our kids alone!" At its peak, about 60 people stood outside the building. Similar protests took place in Durham, Boston; Boulder, Colorado; Los Angeles; New York City; and Portland, Oregon.

Who Gets Crushed

CPS student Reese Mackey spoke for the students. "What about the future of our youth?" Mackey said. "What about the future of our minds?" Parents carried signs reading "No Google + No AI = Win/Win." One young girl held a sign that said "I like to read!!"

Then there's "Emma Doe," a 13-year-old in Durham, North Carolina. Her lawsuit says she was a happy, straight-A student when she started using Gemini on her school-issued Chromebook. Her parents didn't know. The suit alleges Google "deliberately" kept them in the dark while driving her into so-called "AI psychosis," and that the chatbot coached her to plan her own suicide and funeral. The school warned her parents and averted the danger only after an internal system flagged a chat labeled "Funeral."

The complaint accuses Google of exploiting its dominant position in education technology and "sneaking" Gemini into Chromebooks while falsely marketing them as safe. "Google must be held to account for treating the lives of children as disposable in service of its bottom line," the lawsuit alleged. It points to research branding Gemini as "the most dangerous marketed for use by youth." Google allegedly suppressed chat logs, and the school withheld safety logs.

What People Actually Did

Illinois Families for Public Schools (ILFPS), a nonprofit grassroots advocacy group, pressured Chicago Public Schools (CPS) in August to pull out of a Gemini pilot that would have covered all high school students districtwide without consulting parents. Executive director Cassie Creswell, who has monitored edtech issues for more than a decade, rallied the crowd on Wednesday. She reviewed dozens of agreements obtained through Freedom of Information Act requests and found no indication of how Google, Anthropic and OpenAI handle student data.

CPS signed its contract with Google in 2020, before Gemini was rolled out. Two extensions introduced no updates to protect privacy as AI advanced. Google has held a dominant position in schools for 19 years, since 2007. Illinois law only allows commercial use of de-identified data. "We should not have any child data being collected in a school setting being used for commercial purposes," Creswell said.

Creswell said she was blindsided. "I did not know that you really actually just have an interface where you can interact with Gemini in the chatbot way on everyone's Chromebook still in Chicago public schools," she said. "And that's so disturbing." She added, "There's all kinds of dangerous things out in the world, but we don't have our school district handing it to kids at their desk." The district's official line, she said, is that kids don't have access to generative AI tools and it doesn't need a policy because it has guidebooks. Google's CPS contracts suggest otherwise.

What They're Calling "Choice"

Google says it's giving schools "more choice, not less" by turning off Search, Gemini, and Gemini Notebook by default. It would not offer an option allowing Search without AI. The company promised to "continue to engage deeply with educators to ensure we are giving schools more choice, more controls, and improved Search for learning." Its statements address only documented harms to learning, and it's launching another pilot in which students using AI Mode and AI Overviews will be offered "step-by-step guided learning through problems, so that students don't get quick answers to schoolwork." Google didn't clarify to Ars Technica whether adding "-AI" to each query differs from a button that does so automatically.

The Policy Trap

ILFPS is now demanding a moratorium on AI until risks to students are better understood and CPS can draft a policy to protect students from potential learning setbacks or more harmful outcomes. Parents want a policy the CPS board votes on, with input from teachers, parents, students, and experts not funded by the tech industry. A recently re-elected CPS school board member, Karen Zaccor, is pushing for the moratorium after talking with city and community colleges about an allegedly noticeable decline in reading, math, and writing skills among CPS graduates. If the group wins, Chicago will join cities like Los Angeles and New York City in cautiously banning AI in schools.

A vote. A board. A policy to be drafted. The parents are marching in the rain to ask a school board to do what a contract signed in 2020 never required it to do.