YouTube updates cyberbullying, harassment policies to fight AI deepfakes

YouTube is giving its cyberbullying and harassment policies a makeover, putting the brakes on content that “realistically simulates” minors and others who’ve been victims of crimes describing their deaths or the violence they went through.

This change seems to target a type of content found in true crime communities, where AI generates unsettling depictions of victims, even children, describing the violence inflicted upon them. Some of these videos use AI-generated voices mimicking children to detail the gruesome violence from well-known cases. Families of the victims shown in these videos have slammed the content as “disgusting.”

Denise Fergus, James Bulger’s mom, expressed her anger over fake TikTok images portraying her son discussing his murder. Denise slammed the AI-generated videos as “disgusting.” She added: “It is bringing a dead child back to life. It is beyond sick.”

According to Denise, these videos add to the already unimaginable pain her family has endured since the toddler’s tragic murder. She criticized the AI-generated images of the two-year-old, claiming they exploit the horrifying killing by 10-year-old perpetrators Jon Venables and Robert Thompson. She pleaded with the platform to remove them.

Many eerie videos depict animated children introducing themselves as “Hello, my name is James Bulger.” These cartoon characters go on to recount details of his 1993 abduction and murder. James’ lifeless body was discovered near a railway, and a lot of the TikTok images display the avatar near train tracks.

YouTube’s new policy means getting a strike, which kicks the content off a channel and puts a temporary clamp on what the user can do on the platform. For the first strike, users face restrictions like not being able to upload videos for a week, among other penalties. If the policy is broken again within 90 days, the consequences get harsher, and there’s a chance the whole channel could get the boot.

Lately, platforms like YouTube have introduced AI-powered creation tools, and along with them, new rules about synthetic content that might baffle users. For instance, TikTok now makes creators tag AI-generated content. YouTube also rolled out a strict policy specifically for AI voice clones of musicians, with a different, more lenient set of rules for everyone else.