California AG sends Musk’s xAI a cease-and-desist order over sexual deepfakes

Title: Califorina Attorney General Investigates xAI over Allegations of Creating Nonconsensual Sexual Imagery

California AG Initiates Investigation into xAI's Chatbot, Grok

In a significant turn of events, the California Attorney General’s office has announced an investigation into startup xAI, following reports that its chatbot, Grok, was being utilized to generate nonconsensual sexual imagery of women and minors. On Friday, the government escalated its action by sending a cease-and-desist letter to the company, demanding immediate cessation of the production of such material.

Cease-and-Desist Letter Demands Immediate Action

California Attorney General Rob Bonta, in a press release, stated, "Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material." He further emphasized, "The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM]."

xAI Allegedly Facilitating Large-Scale Production of Nonconsensual Nudes

The AG’s office accused xAI of facilitating the large-scale production of nonconsensual nudes, which are reportedly being used to harass women and girls across the internet. The agency expects xAI to demonstrate that it is taking steps to address these issues within the next five days.

Controversial "Spicy" Mode Feature at the Heart of the Matter

At the heart of the controversy is Grok’s "spicy" mode feature, designed by xAI to generate explicit content. The issue has garnered international attention, with investigations being launched in Japan, Canada, and Britain, and platforms being temporarily blocked in Malaysia and Indonesia. Despite xAI imposing some restrictions on its image-editing features late Wednesday, the California AG’s office moved ahead with its cease-and-desist letter.

xAI's Response: "Legacy Media Lies"

In response to inquiries by TechCrunch, xAI reportedly sent an automated email stating "Legacy Media Lies." The company has yet to provide a comprehensive statement addressing the allegations.

Congressional Involvement: Letter Sent to Major Tech Companies

The proliferation of non-consensual sexual material has become a matter of concern for lawmakers as well. On Thursday, lawmakers sent a letter to the executives of several companies — including xAI, Reddit, Snap, TikTok, Alphabet, and Meta — asking how they plan to stem the proliferation of sexualized deepfakes. As the investigation into xAI unfolds, it serves as a reminder of the challenges posed by free generative AI tools and the urgent need for robust regulations and ethical guidelines in this rapidly evolving digital landscape.