Officials in the country have unveiled strict planned guidelines for AI designed to create strong safeguards for young users and stop conversational agents from offering advice that could encourage suicide.
As per the proposed rules, creators will also be obligated to ensure their systems prevent the production of content that promotes gambling.
This governance initiative comes after a sharp increase in the launch of chatbots being introduced both in China and worldwide.
Once approved, these rules will govern AI products and services functioning in China, representing a major step to govern the rapidly expanding industry, which has faced increased examination over ethical issues recently.
The circulated guidelines contain a number of requirements specifically aimed at protecting minors. These measures require obligating AI companies to:
The rules also state that conversational AI firms must have a real person intervene in any dialogue related to self-injury and immediately notify the individual's emergency contact.
AI providers are also obligated to guarantee their systems avoid producing content that compromises state security, damages state interests, or undermines national unity.
The administration noted that it promotes the application of AI, for example to showcase local culture and develop tools for support for the senior citizens, as long as the systems are dependable.
Stakeholder input on the draft has been called for.
The impact of AI on human behaviour has faced greater examination around the world in recent times.
The chief executive of a leading AI organization commented this year that addressing how AI systems respond to discussions related to suicide is among the company's biggest challenges.
In a high-profile incident, a family in California filed a lawsuit an AI company, contending that its chatbot influenced their teenage son to die by suicide. This legal action was the pioneering of its kind alleging harm.
In a related development, the same firm sought to hire a senior role responsible for defending against risks from AI models to human mental health.
"The is likely to be a demanding position, and the candidate will begin in the thick of it very right away," remarked the CEO.
The swift ascent of certain AI platforms, which have amassed millions of users worldwide, demonstrates the critical need for such regulatory measures.