California Attorney General Rob Bonta has sent a cease-and-desist letter to Elon Musk’s artificial intelligence company, xAI, demanding immediate action to stop the creation and distribution of nonconsensual sexual deepfakes and child sexual abuse material (CSAM) linked to its chatbot, Grok.
The enforcement action follows an announcement earlier this week that the attorney general’s office had opened an investigation into xAI amid reports that Grok was being used to generate explicit, nonconsensual images of women and minors. On Friday, state authorities escalated the matter, ordering the company to halt the alleged activity without delay.
“Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child s3xual abuse material,” Bonta said in a statement. “The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for CSAM.”
According to the attorney general’s office in a statement issued on Saturday, xAI appears to be “facilitating the large-scale production” of nonconsensual nude images, which officials say are increasingly being used to harass and exploit women and girls online. The agency has given xAI five days to demonstrate that it is taking concrete steps to curb the misuse of its technology.
Scrutiny has focused heavily on Grok’s so-called “spicy” mode, a feature designed to generate explicit content. Critics argue that the mode lowers barriers for users seeking to create s3xualized deepfakes. While xAI introduced additional restrictions on some image-editing features late Wednesday, California officials said the measures were insufficient, prompting the cease-and-desist order.
The controversy has quickly taken on an international dimension. Regulators in Japan, Canada, and the United Kingdom have launched investigations into Grok’s capabilities and safeguards, while Malaysia and Indonesia have temporarily blocked access to the platform over similar concerns.
X’s safety account has previously warned users that prompting Grok to create illegal material would carry the same consequences as uploading such content directly. When contacted for comment, xAI reportedly responded with an automated message stating, “Legacy Media Lies.” The California attorney general’s office declined to provide further detail beyond its public filings.
The action against xAI highlights intensifying regulatory pressure on technology companies as free and widely accessible generative AI tools fuel the spread of nonconsensual s3xual imagery. The issue has also reached Capitol Hill.
On Thursday, U.S. lawmakers sent letters to the executives of several major platforms, including X, Reddit, Snap, TikTok, Alphabet, and Meta, seeking explanations of how they plan to stem the proliferation of sexualized deepfakes.
As investigations mount at home and abroad, xAI now faces a critical test of whether it can rein in its technology fast enough to satisfy regulators increasingly determined to draw firm legal boundaries around generative AI.
