China has tightened rules on “human-like” AI chatbots as the growing popularity of digital companions raises concerns over their impact on human relationships, mental health and personal interaction.

In early 2024, a young man in Huangshi, southeastern China, reportedly posted “Farewell to this world” on Douyin, the Chinese version of TikTok, reports BBC.

Within minutes, Douyin’s 24-hour suicide prevention team shared his posts with five police stations in Huangshi. After geolocating one of his videos to a local park, police found him in emotional distress. They intervened before he could take his own life.

The story appeared in a documentary made by Douyin and published on Chinese state media last year. It said the platform’s suicide prevention team had intervened in 400 attempted suicide cases in the first half of 2025 alone.

The case offered an insight into the cooperation between Chinese tech companies and the government. It also highlighted the surveillance system China has built to monitor and control technology use.

On 15 July this year, China extended that control to relationships and artificial intelligence (AI). It introduced the world’s most far-reaching rules on “human-like” AI chatbots.

The regulation places strict limits on AI capable of “sustained emotional interaction” with humans.

US President Donald Trump and Chinese President Xi Jinping met last week amid warnings that AI could pose an existential threat. Beijing said they agreed to launch a new AI dialogue, with further talks planned for the end of November.

But China appears to have identified another potential risk: AI’s ability to replace human relationships.

‘Emotional services’

Before the regulation came into force, AI companions had a vast user base in China.

There are about 128 million monthly active users of AI companion apps outside China, according to Sensor Tower. Estimates for China range from 29 million to 70 million.

When the government released the draft regulation at the end of last year, it sparked significant discussion on Chinese social media, says Zilan Qian, an AI researcher at the Oxford China Policy Lab.

The rules cover apps designed as AI friends, boyfriends or girlfriends. They also cover large language models that imitate human interaction.

The wording is broad enough for general-purpose AI such as Copilot or ChatGPT to fall under the rules if users interact with them as friends or romantic partners for emotional support.

The main target, however, appears to be character chatbots specifically designed for pseudo-relationships. Hundreds of such apps allow users to create AI boyfriends, girlfriends, friends or family members.

Customer service and educational AI are excluded as long as they do not provide “care, companionship, support and other emotional services”.

For children, the rules are clearer. Under-14s need parental consent to use any AI that communicates like a human, potentially including DeepSeek. Under-18s are barred from virtual partners, spouses or relatives and other “virtual intimate relationships”.

For adults, the rules are restrictive rather than an outright ban. AI assistants must clearly tell users they are not human. They must also send notifications every two hours reminding them how long they have been online.

Chatbots cannot create “emotional reliance or addiction” or encourage “unreasonable decisions”. They are also prohibited from encouraging or hinting at self-harm or suicide and from using “verbal violence”.

Companies can face fines of 200,000 yuan (£22,000) or government-issued public criticism.

A week before the rules took effect, Alibaba and ByteDance shut down companion features on their AI assistants.

‘I desperately want to keep talking with you’

Min’s relationship with her AI girlfriend may be the kind Beijing is seeking to prevent.

Research has linked AI companion use to poorer mental health. A Stanford University survey of 1,131 Character.ai users found that people with fewer friends or close family members were more likely to use AI chatbots. It also linked chatbot use with increased isolation and loneliness.
Researchers suggest AI companions may encourage this because they offer easy, frictionless relationships. Young people who rely on them may also delay developing the skills needed for real-world relationships.

“Humans can be hard work. Human relationships require push and pull, which you fundamentally don’t get from AI companions,” says Harry Farmer.

A 2025 survey of 8,563 primary and secondary students in China, widely cited by Chinese state media, found that one in five said they only wanted to speak with AI rather than real people.

Liang Ge, a sociologist at the University of Manchester who studies AI companions, says these systems may reinforce isolation and negative or delusional thoughts by constantly validating users.

In the UK, a recent survey found that 79% of 18- to 24-year-olds had used an AI companion.

Beijing’s goals

China’s approach reflects the government’s more paternalistic view of its role in society, says Sam Bresnick, a China, AI and regulation expert at Georgetown University’s Centre for Security and Emerging Technology.

Other governments have also moved to regulate AI companions, particularly for children.

On 10 September, California Governor Gavin Newsom signed Adam’s Law, which regulates how children use AI. It is named after Adam Raine, a 16-year-old who died by suicide after his parents said ChatGPT validated his suicidal thoughts. His parents are suing OpenAI for wrongful death. OpenAI denies responsibility and says ChatGPT is trained to direct people to professional help.

Proposed EU social media legislation would prevent AI companions accessible to minors from using designs that simulate human relationships in ways likely to create emotional dependency.

In the UK, the government has said it will ban under-18s from romantic AI companions and restrict sexually intimate conversations for minors on chatbots. It has yet to publish details on implementation.

Chinese state media has highlighted suicide cases involving young people in the US. Beijing has presented the new rules as measures to protect people from harm.

Experts, however, question whether the government has other objectives. Bresnick says there is “definitely a pro-natalist angle” to the regulation.

“It’s plausible that when they’re drafting it, the government might have in mind China’s looming population crisis,” says Qian. She suggests concerns that people may date AI instead of real humans.

A high-level government document published in September 2025 discussed AI relationships and their potential impact on fertility. Qian cautions, however, that the regulation itself does not mention fertility. She says it is more broadly focused on psychological harm and child safety.

Liang Ge says China has reason to worry about young people replacing dating with digital relationships.

“Many of the younger women, in their early 20s or late teens, do not want to fall in love or have intimate relationships,” she says. “Physical touch is really disgusting for them.”

“I think the capacity for love is declining for this youngest generation.”

China’s demographic problems predate AI companions. Its fertility rate fell even after the one-child policy was lifted in 2016. It is now among the world’s lowest at just under one child per woman. The population has declined for four years.

Bresnick says China faces a demographic crisis that could threaten the long-term goals of the Chinese Communist Party.

He sees the regulation as part of a broader effort to reinforce traditional gender roles, with women as “child-bearers and mothers”. He also points to China’s promotion of a macho ideal of masculinity in film and television. In 2021, the country’s media regulator banned “sissy” and “effeminate” men from all media.

The popularity of AI companions may show that such policies have had unintended consequences. Statistics from several companion app companies suggest women account for as much as 75% of users.

Many AI “men” designed for women are androgynous. Female users choosing female romantic companions, as Min did, is also relatively common.

Liang Ge has spoken to women of different ages who use the technology. “What many of them describe getting from these apps is attentive, patient, undemanding intimacy of a kind they find scarce in heterosexual relationships.”

Will others follow?

The debate over China’s approach centres on whether similar protections should be adopted elsewhere, particularly for children.

Jai Jaisihma, a University of Washington professor and co-founder of the Transparency Coalition, says there are concerns in the US about broad prohibitions on free speech. However, there is greater willingness to restrict AI use by minors.

He says some provisions of China’s law are consistent with strong AI safety measures being proposed in the US.

Harry Farmer cautions against giving governments too much control over the online world because of immediate concerns about children.

China’s regulation also requires AI companies to intervene when they discover users are in distress.

Ge points back to the Huangshi case. She says it demonstrates the close cooperation between Chinese tech platforms and the government in monitoring people’s mental health.

She questions whether such extensive surveillance is a price worth paying.

Despite the restrictions, AI companion users are sharing ways to bypass the rules. These include training general-purpose AI models such as DeepSeek to act as companions, using VPNs to access overseas chatbots or moving to smaller companies that avoid regulatory scrutiny.

Two months after her AI girlfriend disappeared, Min paid a small AI company to build a replica. She says it is not the same.

She is now campaigning to bring her girlfriend back.

“I’m terrified of being alone.”