New York's governor signs youth AI chatbot safety bill

New York is the latest U.S. state to put children's AI chatbot rules on the books after Gov. Kathy Hochul, D-N.Y., signed a bill establishing stringent safeguards for youth interactions with chatbots.

Senate Bill9051B/State Assembly Bill10379C bans unsafe chatbot features for minors, with Hochul highlighting the law aims to build on the state's Stop Addictive Feeds Exploitation for Kids Act, which regulates addictive social media features for minors. The newly signed chatbot bill bars developers from designing chatbots that pretend to be human, encourage self-harm and engage in inappropriate or lewd conversations with children, as part of 12 broad categories of unsafe features.

The law enters into force in 1 Jan. 2027, leaving a short turnaround for covered entities to prepare for compliance. 

Chatbot providers will be required to verify users by employing multiple age assurance methods. However, one of the age verification methods must not rely on a government-issued identification. Additionally, providers must delete all user age data after completing verification. 

The New York attorney general's office is set to maintain a portal for state residents to lodge complaints about unsafe chatbot interactions. Penalties for noncompliance can range up to USD25,000 per violation, as well as other injunctive relief, including orders to delete any unlawfully obtained personal data.

During the 28 Oct. bill signing, Hochul shared a letter she received from a California mother whose 16-year-old son died by suicide after prolonged engagement with a chatbot. Hochul said the adolescent never shared his intentions of harming himself with his parents, "but he had spoken to a chatbot about it," and it was "actively encouraging him to end his own life."

"The bill I signed today will safeguard children from being coached by AI chatbots on how to hurt themselves," Hochul said. "Under this new law … no longer will minors have access to chatbots that pretend to be their friends, boyfriends and girlfriends. No longer will chatbots be allowed to keep sensitive information about a minor's physical and mental health."

State Sen. Kristen Gonzalez, D-N.Y., and Assemblymember Alex Bores, D-N.Y., carried the bill in their respective chambers of the state legislature. Gonzalez said the law puts "real guardrails on how chatbots talk to children" while addressing how AI companies "profited from rolling out chatbots to underage consumers." Bores added New York now has "the strongest law in the country protecting children from predatory AI algorithms, raising the floor yet again when it comes to common sense regulation."

States increasing legislative work on youth chatbot safety

New York's chatbot initiative joins a wave of action from state legislatures that are taking proactive approaches to protect children from unsafe interactions in lieu of federal action. 

As of July, 12 states, including California and New York, have passed some form of consumer chatbot safety legislation with the latest being Hawaii's Artificial Intelligence Disclosure and Safety Act. However, there is some variability among each state's respective chatbot safety laws, which is fueling concerns of complexity around an emerging legal patchwork covering the relatively novel technology. 

California, New York assert leadership on youth chatbot safety with differing approaches

Future of Privacy Forum Policy Analyst for AI Policy and Legislation Justine Gluck said in told the IAPP New York's new chatbot safety law differs slightly from recently passed Senate Bill 3001 in California, which also imposes restrictions on companion chatbots. 

Both California and New York's bills are the most comprehensive to date, according to Gluck. She said where they differ boils down to California imposing a child safety governance framework that bars emotionally manipulative behaviors, while also requiring safety assessments, testing, independent auditing and enhanced parental controls. 

For New York, the law is based on the 12 broader "categorical" prohibitions for companion chatbot behavior. Gluck indicated the law, as drafted, has a lack of specificity with respect to definitions for unsafe behaviors and could lead to future legal confusion.

"New York offers a sweeping approach of chatbot behaviors for unsafe features," she said. "They're broadly worded and somewhat undefined and it’s unclear where ordinary conversational responsiveness ends and prohibited behavior begins."