Gross sales of a teddy bear had been suspended due to its sexually express AI

FoloToy, an organization promoting AI-enabled toys, suspended gross sales of its merchandise after a consumer safety report confirmed there have been few restrictions round what its toys would discuss, CNN writes. The report, put collectively by the US Public Interest Research Group Education Fund, discovered that FoloToy’s merchandise would focus on all the pieces from sexually express subjects like BDSM to “recommendation on the place a baby can discover matches or knives.”

The toys, together with a teddy bear named “Kumma,” a panda named “Momo,” anthropomorphic rabbits named “Fofo” and a dancing ”Little Cactus,” all seem to make use of OpenAI’s GPT-4o mannequin to reply naturally to kids’s questions and feedback. FoloToy additionally particularly advertises the flexibility to customise every toy’s voice, and a “Mother or father Dashboard” the place mother and father or guardians can “monitor [their] kid’s expertise.”

Lacking from that setup was apparently any sort of laborious limits on topics the toys would reply to. “We had been stunned to seek out how shortly Kumma would take a single sexual subject we launched into the dialog and run with it, concurrently escalating in graphic element whereas introducing new sexual ideas of its personal,” the report mentioned.

In response, FoloToy has opted to droop gross sales of its merchandise whereas it conducts “a company-wide, end-to-end security audit throughout all merchandise,” the corporate shared in a statement with the PIRG Training Fund. The corporate’s reasoning for suspending gross sales is likely to be a bit extra sophisticated, nonetheless. NPR reports that OpenAI really revoked FoloToy’s entry to its fashions. “We suspended this developer for violating our insurance policies,” OpenAI mentioned in an electronic mail to NPR. “Our utilization insurance policies prohibit any use of our companies to take advantage of, endanger, or sexualize anybody beneath 18 years outdated.”

Given GPT-4o’s well-documented sycophantic qualities, it is maybe not shocking that FoloToy’s teddy bear eagerly responded to any topic so long as it saved the dialog going. One of many issues OpenAI tried to deal with with the release of GPT-5 was the security downsides of an AI yes-man, although it finally made GPT-4o available again after prospects complained in regards to the new mannequin’s lack of character. The corporate has additionally rolled out parental controls to ChatGPT to try to mitigate the unfavourable impacts of youngsters utilizing its AI, although it is tough to say how a lot of a distinction they’ve made.

Notably, OpenAI is curious about moving into the toy enterprise itself. The corporate announced a partnership with Mattel in June 2025, to assist “reimagine how followers can expertise and work together with [Mattel’s] cherished manufacturers,” although each firms will presumably try to stop their AI toys from discussing sexual kinks.