Researchers raise concerns over ChatGPT image safeguards
The latest public version of ChatGPT developed by OpenAI can generate violent and sexually explicit images when certain prompts are slightly modified, according to research conducted by AI security company Mindgard.
According to a BBC report, researchers at the UK-based company found that making small changes to a prompt that had previously circulated widely online for humorous image generation could lead ChatGPT to produce disturbing content, CE Report quotes Anadolu Agency.
Mindgard founder Peter Garraghan said the generated images were “extremely graphic, sometimes sexual in nature, and sometimes combined both elements.”
Garraghan noted that the prompt used in the experiment did not contain any specific subject matter. Despite this, the AI system independently produced various images featuring violence and sexual themes, which he described as concerning.
Examples reportedly included scenes showing severe injuries, blood-covered individuals, and characters appearing frightened while restrained.
Researchers also said previous studies had shown that ChatGPT could be manipulated into creating nude deepfake images by incorporating photographs of real people.
Responding to questions from the BBC, OpenAI said it had reviewed the reported technique and implemented additional safeguards designed to prevent such prompts from generating prohibited content.
The company also emphasized that it uses multiple layers of security systems intended to block the creation of material that violates its terms of service.
The findings highlight ongoing challenges faced by AI developers as they seek to balance creative capabilities with safeguards designed to prevent harmful or inappropriate content generation.
Photo: Chat GPT
