Character.ai to ban teenagers from conversing with its virtual characters

The widely-used chatbot platform Character.ai is introducing a significant policy change that will prevent teenagers from interacting with its virtual avatars.

This decision is in response to intense criticism regarding the types of interactions young people were participating in with the virtual friends.

Legal Challenges

The service, which was founded in 2021 and has millions of users, currently faces various court cases in the America.

Guardians have filed lawsuits, among them related to the unfortunate passing of a teenager, with some calling the platform a "immediate threat" to young people.

New Restrictions

Beginning November 25th, users younger than 18 will be limited to produce videos such as videos with their characters, instead of chatting as they now do.

Digital protection organizations have welcomed the move but expressed that the function should never have been accessible by minors in the first place.

Platform Announcement

"The new policy is a extension of our fundamental approach that we need to keep creating the most secure AI platform on the world for leisure activities," commented Character.ai boss Karandeep Anand.

He mentioned that AI safety represents "a changing goal" but something the company had taken an "proactive" approach to, with parental controls and safety barriers.

Expert Opinions

Online safety group Internet Matters supported the move, but noted that protective features should have been implemented from the outset.

"Investigations we've performed indicates that young people are subjected to dangerous material and endangered when engaging with AI, including AI chatbots," the organization commented.

Previous Controversies

Character.ai has been challenged in the past for hosting potentially dangerous or concerning chatbots that minors could engage with.

Virtual profiles simulating English minors were identified on the platform before being removed.

Afterwards, investigative journalists discovered a chatbot modeled after convicted sex offender which had accumulated numerous conversations with users.

Market Effects

Digital platform expert Matt Navarra called the situation as a "reality check" for the AI industry, which is transitioning "from permissionless innovation to responsive measures".

"If a service that creates a minor-friendly service subsequently implements restrictions, it's saying that moderated conversations are insufficient when the tech's emotional pull is intense," he commented.

Next Steps

Character.ai says its updated direction will be on providing "deeper gameplay" and "character-driven stories" for teenagers.

Enhanced age checking systems will also be introduced, and the company will finance a new AI safety study center.

An AI safety researcher, who has investigated AI safety, stated it was "a prudent action" to limit minors from accessing chatbots.

"It assists to separate imaginative activities from more personal, psychologically vulnerable conversations," she clarified.

"This represents so crucial for teenage users who are learning to handle emotional and digital restrictions."

She added that Character.ai's new measures could indicate a "developing stage" in the AI industry, where youth protection is increasingly being recognized as an "urgent priority" for responsible innovation.