Deepfake, AI or real? It’s getting harder for police to protect children from sexual exploitation online

Artificial intelligence (AI), now an integral part of our everyday lives, is becoming increasingly accessible and ubiquitous. Consequently, there’s a growing trend of AI advancements being exploited for criminal activities.

One significant concern is the ability AI provides to offenders to produce images and videos depicting real or deepfake child sexual exploitation material.

This is particularly important here in Australia. The CyberSecurity Cooperative Research Centre has identified the country as the third-largest market for online sexual abuse material.

So, how is AI being used to create child sexual exploitation material? Is it becoming more common? And importantly, how do we combat this crime to better protect children?

Spreading faster and wider

In the United States, the Department of Homeland Security refers to AI-created child sexual abuse material as being:

the production, through digital media, of child sexual abuse material and other wholly or partly artificial or digitally created sexualised images of children.

The agency has recognised a variety of ways in which AI is used to create this material. This includes generated images or videos that contain real children, or using deepfake technologies, such as de-aging or misuse of a person’s innocent images (or audio or video) to generate offending content.

Deepfakes refer to hyper-realistic multimedia content generated using AI techniques and algorithms. This means any given material could be partially or completely fake.

The Department of Homeland Security has also found guides on how to use AI to generate child sexual exploitation material on the dark web.

The child safety technology company Thorn has also identified a range of ways AI is used in creating this material. It noted in a report that AI can impede victim identification. It can also create new ways to victimise and revictimise children.

Concerningly, the ease with which the technology can be used helps generate more demand. Criminals can then share information about how to make this material (as the Department of Homeland Security found), further proliferating the abuse.

How common is it?

In 2023, an Internet Watch Foundation investigation revealed alarming statistics. Within a month, a dark web forum hosted 20,254 AI-generated images. Analysts assessed that 11,108 of these images were most likely criminal. Using UK laws, they identified 2,562 that satisfied the legal requirements for child sexual exploitation material. A further 416 were criminally prohibited images.

Similarly, the Australian Centre to Counter Child Exploitation, set up in 2018, received more than 49,500 reports of child sexual exploitation material in the 2023–2024 financial year, an increase of about 9,300 over the previous year.

About 90% of deepfake materials online are believed to be explicit. While we don’t exactly know how many include children, the previous statistics indicate many would.

These data highlight the rapid proliferation of AI in producing realistic and damaging child sexual exploitation material that is difficult to distinguish from genuine images.

This has become a significant national concern. The issue was particularly highlighted during the COVID pandemic when there was a marked increase in the production and distribution of exploitation material.

This trend has prompted an inquiry and a subsequent submission to the Parliamentary Joint Committee on Law Enforcement by the Cyber Security Cooperative Research Centre. As AI technologies become even more advanced and accessible, the issue will only get worse.

Detective Superintendent Frank Rayner from the research centre has said:

the tools that people can access online to create and modify using AI are expanding and they’re becoming more sophisticated, as well. You can jump onto a web browser and enter your prompts in and do text-to-image or text-to-video and have a result in minutes.

Making policing harder

Traditional methods of identifying child sexual exploitation material, which rely on recognising known images and tracking their circulation, are inadequate in the face of AI’s ability to rapidly generate new, unique content.

Moreover, the growing realism of AI-generated exploitation material is adding to the workload of the victim identification unit of the Australian Federal Police. Federal Police Commander Helen Schneider has said

it’s sometimes difficult to discern fact from fiction and therefore we can potentially waste resources looking at images that don’t actually contain real child victims. It means there are victims out there that remain in harmful situations for longer.

However, emerging strategies are being developed to address these challenges.

One promising approach involves leveraging AI technology itself to combat AI-generated content. Machine learning algorithms can be trained to detect subtle anomalies and patterns specific to AI-generated images, such as inconsistencies in lighting, texture or facial features the human eye might miss.

AI technology can also be used to detect exploitation material, including content that was previously hidden. This is done by gathering large data sets from across the internet, which is then assessed by experts.

Collaboration is key

According to Thorn, any response to the use of AI in child sexual exploitation material should involve AI developers and providers, data hosting platforms, social platforms and search engines. Working together would help minimise the possibility of generative AI being further misused.

In 2024, major social media companies such as Google, Meta and Amazon came together to form an alliance to fight the use of AI for such abusive material. The chief executives of the major social media companies also faced a US senate committee on how they are preventing online child sexual exploitation and the use of AI to create these images.

The collaboration between technology companies and law enforcement is essential in the fight against the further proliferation of this material. By leveraging their technological capabilities and working together proactively, they can address this serious national concern more effectively than working on their own.

The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.