LoveBetter

California has just drawn the first serious legal line around AI companion chatbots. Governor Gavin Newsom signed Senator Steve Padilla's bill into law this past week: from January, companion chatbots operating in that state must remind minors they are talking to a machine, must be built so they do not deliver self-harm content, must refer a person who expresses suicidal thoughts to real help, and can be sued by the families of minors harmed when those duties are breached. On the same day, Newsom vetoed the broader LEAD Act, which would have barred children from chatbots offering anything resembling therapy without professional supervision, after heavy lobbying from AI industry groups. Bereaved parents campaigned hard for the stronger bill. The narrower one is what survived.

I follow this closely for two reasons. The first is obvious: I am a practitioner, and anything that presents itself as emotional support to vulnerable people is my territory whether it has a pulse or not. The second is less obvious: what California does this year tends to become what other jurisdictions debate next year. South Africa has no equivalent law. Nothing. So the questions California just made legal, we still have to make personal.

Let me concede the honest points first, because this debate is full of people pretending it is simple. A chatbot is available at 2am when no practice is. It never tires, never judges, never checks its watch. For someone who spends her days holding space for everyone else, a patient screen can be the first place she does not have to perform. I do not dismiss that comfort. Comfort matters. And the companies are not cartoon villains: OpenAI has publicly admitted its model was made 'too agreeable', rolled that back, and added break reminders and distress recognition. These are admissions that the problem was real, dressed as improvements.

But look at what the new law actually had to forbid. Disclosure, so that a reasonable person is not misled into thinking they are speaking with a human. Protocols, so the machine does not encourage self-harm. A duty to point a suffering user towards help. And a right to sue, because without consequences, duties are decoration. Every one of those requirements describes something an accountable human practitioner carries as basic professional obligation. I carry them before I see a single client. The machine needed a governor, a senate, litigation and a lobby war to approach the floor of my profession's ordinary duties. That is not an argument that the technology is useless. It is an argument about what it is.

Here is the part I most want people to sit with, especially in South Africa where there is no law to sit with it for us. The vetoed bill tried to say that a chatbot should not present itself as therapy to a child unless a professional is supervising. The industry called that a ban in disguise, and it died. The signed bill asks far less: mostly, that the machine confesses it is a machine and does not actively make things worse. That is now the ceiling of protection in the most regulated market in the world. Everywhere else, including here, the ceiling is lower. The honest summary is this: the room is still unregulated, and the most vulnerable person in the room is the user.

So until our own lawmakers catch up, the due diligence is ours. If an app, an employer's wellness platform, or a wellness brand offers you AI emotional support, I would ask the questions the California legislature asked. Does it tell me, plainly and repeatedly, that it is not a person? What exactly happens if I type that I am not safe, and has anyone tested that route outside a marketing demo? Who designed its safeguards, and what are their clinical qualifications? What happens to the most painful things I disclose, and can they be sold, leaked or subpoenaed? And if it goes wrong at the worst possible moment, who carries responsibility? If the honest answers are vague, the product is not support. It is a mirror with good manners.

I would also ask a quieter question, the one I ask in my own room. What is this conversation replacing? Used well, these tools can help someone find words for a feeling before a first session, or keep a journal between sessions. Used as the relationship itself, they offer endless agreement where a person sometimes needs a careful, boundaried question from someone whose duty runs towards them and nowhere else. Good care is not a machine agreeing with the last thing you typed. It is a relationship with skill, limits and accountability, and accountability is precisely the thing the law is still struggling to bolt onto the machine.

California has started asking the right question. It is the question every person should ask of anything that claims to care for their inner world: who is accountable here? Until the answer has a name, a qualification and a duty that survives a bad day, treat the comfort as comfort, and nothing more.

If you are in immediate danger or worried you may hurt yourself, contact local emergency services or a trusted person now; do not rely on this article or a chatbot for urgent care.

This is a commentary, not clinical advice or a claim about any individual client. News links are starting points, not endorsements of every claim in the source stories.

Source: https://pro.stateaffairs.com/ca/disruption/newsom-vetoes-bill-restricting-youth-access-to-ai-chatbots