Child Safety

AI child safety risks are real and need safeguards. Thankfully, federal law already forbids illegal content generated by an AI and models already implement strong protections.

Is AI CSAM illegal?

• Yes, the FBI explicitly states that AI child sexual abuse material (CSAM) is illegal.¹

• Long-standing federal criminal statutes apply fully to AI-generated content—no new regulations are required to make AI-generated CSAM illegal.

• The federal government is already prosecuting criminals who use AI to generate illicit materials.²

What has the government done to address deepfakes?

• The TAKE IT DOWN Act prohibits sexual exploitation from AI deepfakes.³

  • Under this regulation, platforms are required to take down the content when requested.

• AI is helping the government to better help identify victims and catch offenders.⁴

How do AI companies prevent CSAM generation?

• The top AI companies—OpenAI,⁵ Anthropic,⁶ Google,⁷ Meta,⁸ and xAI⁹—explicitly prohibit the generation of AI CSAM in their policies.

• Organizations such as Thorn have developed frameworks and solutions to help AI companies minimize CSAM risks.¹⁰

• Many leading AI companies have committed to specific “Safety by Design” principles to protect child safety.¹¹

---

Additional Resources

Public service announcement on AI-generated CSAM from the FBI.
This public service announcement from the FBI reiterates that AI-generated CSAM is strictly illegal.

“Artificial Intelligence and Combatting Online Child Sexual Exploitation and Abuse” from the Department of Homeland Security.
This flyer from the DHS highlights the illegality of AI-generated CSAM and the opportunities for law enforcement to use AI to help identify victims and catch offenders.

TAKE IT DOWN Act from the US Congress.
This law prohibits exploitation from AI-generated deepfakes and requires platforms to take the content down.

“Safety by Design: One year of progress protecting children in the age of AI” from Thorn.
This article discusses the work by Thorn to help AI companies operate with “Safety by Design” principles that protect children.

OpenAI safety policy: Explicitly prohibits AI CSAM.

Meta safety policy: Explicitly prohibits AI CSAM.

Google safety policy: Explicitly prohibits AI CSAM.

Anthropic safety policy: Explicitly prohibits AI CSAM.

xAI safety policy: Explicitly prohibits AI CSAM.

1. Federal Bureau of Investigation, “Child Sexual Abuse Material Created by Generative AI and Similar Online Tools is Illegal” March 29, 2024.

2. “Hilliard man pleads guilty to using AI to create child pornography,” United States Attorney’s Office, December 1, 2025.

3. “TAKE IT DOWN Act,” US Congress.

4. Department of Homeland Security, “Artificial Intelligence and Combatting Online Child Sexual Exploitation and Abuse.”

5. OpenAI, “Usage Policies.”

6. Anthropic, “Usage Policy.”

7. Google “Generative AI Prohibited Use Policy.”

8. Meta, “Child Sexual Exploitation, Abuse, and Nudity.”

9. xAI, “xAI Acceptable Use Policy.”

10. Thorn, “Protect your AI model from misuse.“

11. Thorn, “Thorn and All Tech Is Human Forge Generative AI Principles with AI Leaders to Enact Strong Child Safety Commitments,” July 16, 2024.