Malaysia will take legal action against Elon Musk’s social media platform X and its artificial intelligence unit xAI, according to a statement from Malaysia’s communications regulator. The regulator said it has identified misuse of the Grok chatbot to generate and distribute harmful content, and that it has previously served notices to the companies to remove that content without effect.
The announcement comes days after Malaysia and Indonesia blocked access to Grok, as concerns have grown that the chatbot is being misused to generate sexually explicit and non-consensual images.
In its statement, the Malaysian Communications and Multimedia Commission said it has identified the misuse of Grok to produce and share harmful material that includes sexually explicit, indecent, extremely offensive, and non-consensual manipulated images. The commission said it served notices to X and xAI earlier this month to remove the harmful content, but that “no action has been taken.”
The commission also said it has appointed a lawyer and that legal proceedings would begin soon. It added: “Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments”.
The regulator’s move reflects wider scrutiny of generative AI tools that can produce realistic images, sound and text, along with concern that existing safeguards may not be preventing abuse. Grok, which launched in 2023, is free to use on X, and its image generator feature Grok Imagine was added last year with a so-called spicy mode that the regulator said can generate adult content.
Grok has faced criticism for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. The scrutiny has broadened beyond Southeast Asia, with the European Union, India and the United Kingdom all facing pressure over how such tools handle sexually abusive content.
The United Kingdom has said it is moving to criminalize “nudification apps,” and Britain’s media regulator launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes, but critics said the change did not fully address the problem.
Musk and his companies have not publicly commented on the restrictions in Malaysia and Indonesia. In the absence of a public response, xAI was reported to have issued an automated reply to media queries saying, “Legacy Media Lies.”