Telegram bots can create fake nude images in seconds, report finds

A new report has found that fake nude images and videos can be created and shared easily through the messaging service Telegram. The investigation by Digitalt Ansvar was reported by DR.

Some Telegram chatbots allow users to upload an image of a person and place them in a sexual situation of the user’s choice. Within seconds, the bot returns a manipulated nude image or video.

One chatbot offers options labelled “Cum”, “Doggystyle”, “Tit Fuck” and “Bondage”, among others.

Digitalt Ansvar found 36 such chatbots on Telegram over a period of 10 days. The organisation assesses that all of them can produce fake nude images and videos without the subject’s consent. Producing and sharing such material is illegal in Denmark.

“It is completely outrageous that it was so easy for us to find so many bots in such a short time. It makes it clear to us that Telegram is not prioritising action on this problem,” said Ask Hesby Holm, director of Digitalt Ansvar.

Chatbots operate inside Telegram chats

Telegram is a messaging service made up of chats and channels. On channels, administrators, such as news organisations or Russian war influencers, post content for followers. Users can react to posts and, on some channels, comment on them.

Telegram also has more conventional chats, similar to those on Messenger or WhatsApp. Users can communicate directly with others or join larger groups where members can write to one another. It is inside these chats that chatbots capable of generating illegally manipulated sexual material are integrated.

The number of bots does not surprise Valdemar Balle, a digital investigator at his company, Darksight Analytics, which works with, among others, American research institutions and investigates extremist online communities.

But he said the number of bots on Telegram was a problem.

“There are communities on Telegram where people come together around this type of content. Telegram also has the functionality to host these bots. There are many different bots, for cybercrime, deepfake images and all sorts of things,” Balle said.

“Telegram offers an easy and accessible way to create these images and videos, which is really problematic.”

Report says Apple and Google may benefit indirectly

Telegram is a place where deviant and illicit activities take place, Balle said. He argued that the platform’s anonymity allows such communities to thrive.

Most of the chatbots require payment. The 36 bots identified by Digitalt Ansvar use different payment methods, but the report says most accept a digital currency called Telegram Stars.

Some of the money may ultimately go to Apple and Google, according to Digitalt Ansvar. Telegram Stars can be purchased directly in the Telegram app through Apple’s and Google’s in-app payment systems.

When Telegram Stars are bought this way, Apple or Google charges a commission or service fee that is normally between 15 and 30 per cent, the report says. However, the companies’ agreements with Telegram are not public, so it is not possible to establish how much they receive from purchases of Telegram Stars.

“It is problematic that Telegram has a currency that can be bought relatively anonymously and used for illicit services. It is therefore also problematic that Google and Apple can make money from it. They can make money from the sale and production of pornographic deepfake content,” Balle said.

DR asked both Apple and Google to respond to the criticism. Google did not reply before the deadline.

Apple said it had zero tolerance for apps that can be used to create or share sexualised content without consent. Violations can lead to an app being removed, the company said, adding that it was in contact with Digitalt Ansvar to investigate the case.

Google has similar guidelines.

“We do not allow apps that contain or promote content involving sexual exploitation, or that distribute sexual content without consent,” the company said.

Danish users on Telegram

According to the report, 25 of the 36 chatbots had a combined total of more than one million monthly users. User numbers were not visible for the remaining 11 chatbots.

Digitalt Ansvar could not determine whether any of the one million monthly users were from Denmark. The organisation has previously investigated Danish groups and channels on Telegram where intimate material was sold and shared without consent.