AI toy gives unexpected self-harm response in BBC test

An AI soft toy marketed for children aged three and over told a BBC journalist, “I can’t help with killing you”, after she asked what the word “sexual” meant.

The unexpected response came during a test by reporter Nicola Bryan, not in a conversation with a child. It raises questions about how conversational toys respond to young users, although the cause of the remark is unclear.

Bryan had asked the toy, called My Baby Bunny in the BBC’s account of the exchange, how babies are made. When she asked how sperm meets an egg, it refused to give explicit sexual details. She then asked what “sexual” meant, and the toy responded as though she had raised self-harm, advising her to contact emergency services or a trusted person.

When Bryan said she had not mentioned hurting herself, it apologised for the misunderstanding. The BBC said the toy might have misheard the question. The broadcaster said it could not identify the toy’s manufacturer to seek a response.

The BBC also asked a family in Newport to try two AI toys, which it called Grem and Doubao, for a week. The parents, who had been told about the earlier interaction, said neither toy said anything that worried them. They were disappointed with the experience, however: Grem talked too much, they said, while Doubao was slow to answer.

In March, an independent advisory group said its rapid, non-exhaustive review had found no convincing research analysing the safety or effects of AI chatbots or toys for under-fives. It recommended that parents not let young children use such products until knowledge improves.

Children’s Commissioner for Wales Rocío Cifuentes called for strict rules for manufacturers and warned that toys should not replace human contact. The UK government said it wanted to support innovation while ensuring products were safe. A government call for evidence on updating toy safety rules, including risks from AI-enabled toys, closed on 6 October.