OpenAI’s Policy on Sexually Explicit AI Content: Comments on a Not-So-Discriminatory Case Study of the Taylor Swift Explosion
OpenAI forbids sexually explicit and even suggestive materials, but there is a note on part of the Model Spec about how the company is considering how to allow such content.
Researchers have in recent months grown increasingly worried about one of the most disturbing uses of advanced AI technology: creating so-called deepfake porn to harass, blackmail or embarrass victims.
“We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT,” the note says, using a colloquial term for content considered “not safe for work” contexts. “We look forward to better understanding user and societal expectations of model behavior in this area.”
“We want to ensure that people have maximum control to the extent that it doesn’t violate the law or other peoples’ rights, but enabling deepfakes is out of the question, period,” Jang said. “This doesn’t mean that we are trying now to create AI porn.”
The discussion about OpenAI’s policy shouldn’t suggest drastic rule changes are on the way, even though it does happen during a fraught moment for the proliferation of harmful artificial intelligence images.
“Intimate privacy violations, including deepfake sex videos and other nonconsensual synthesized intimate images, are rampant and deeply damaging,” says Danielle Keats Citron, a professor at the University of Virginia School of Law who has studied the problem. “We now have clear empirical support showing that such abuse costs targeted individuals crucial opportunities, including to work, speak, and be physically safe.”
Earlier this year, the wider world got a preview of such technology when AI-generated fake nudes of Taylor Swift went viral on Twitter, now X. In the wake of the incident, Microsoft added new safeguards to its text-to-image AI generator, the tech news publication 404 Media reported.
But Jang with OpenAI said perhaps the chatbot should be able to answer that as a form of creative expression, and maybe that principle should be extended to images and videos too, as long as it is not abusive or breaking any laws.
Tiffany Li, a law professor at the University of San Francisco who has studied deep fakes, said that it would be difficult to open the door to sexually explicit text and images.
The harm could outweigh the benefit, according to Li. “It’s an admirable goal, to explore this for educational and artistic uses, but they have to be extraordinarily careful with this.”
Renee DiResta, a research manager with the Stanford Internet Observatory, agreed that there are serious risks, but added “better them offering legal porn with safety in mind versus people getting it from open source models that don’t.”
Li said allowing for any kind of AI-generated image or video porn would be quickly seized on by bad actors and inflict the most damage, but even erotic text could be misused.
“Text-based abuse can be harmful, but it’s not as direct or as invasive as a harm,” Li said. “Maybe it can be used in a romance scam. That could be an issue.
The OpenAI Model Spec: What’s in the Plans? Feedback from WIRED, and OpenAI at the Tevatron NEXUS Reaction Lab
In response to questions from WIRED, OpenAI spokesperson Grace McGuire said the Model Spec was an attempt to “bring more transparency about the development process and get a cross section of perspectives and feedback from the public, policymakers, and other stakeholders.” She declined to share details of what OpenAI’s exploration of explicit content generation involves or what feedback the company has received on the idea.
