OpenAI said the service would empower businesses and startups and granted Microsoft, a hefty backer of OpenAI, an exclusive license to the underlying algorithms. WIRED and some coders and AI researchers who tried the system showed it could also generate unsavory text, such as anti-Semitic comments, and extremist propaganda. OpenAI said it would carefully vet customers to weed out bad actors, and required most customers—but not Latitude—to use filters the AI provider created to block profanity, hate speech, or sexual content.
Out of the limelight, AI Dungeon provided relatively unconstrained access to OpenAI’s text-generation technology. In December 2019, the month the game launched using the earlier open-source version of OpenAI’s technology, it won 100,000 players. Some quickly discovered and came to cherish its fluency with sexual content. Others complained the AI would bring up sexual themes unbidden, for example when they attempted to travel by mounting a dragon and their adventure took an unforeseen turn.
Latitude cofounder Nick Walton acknowledged the problem on the game’s official Reddit community within days of launching the game. He said several players had sent him examples that left them “feeling deeply uncomfortable,” adding that the company was working on filtering technology. From the game’s early months players also noticed, and posted online to flag, that it would sometimes write children into sexual scenarios.
AI Dungeon’s official Reddit and Discord communities added dedicated channels to discuss adult content generated by the game. Latitude added an optional “safe mode” that filtered out suggestions from the AI featuring certain words. Like all automated filters, however, it was not perfect. And some players noticed the supposedly safe setting improved the text-generator’s erotic writing because it used more analogies and euphemisms. The company also added a premium subscription tier to generate revenue.
When AI Dungeon added OpenAI’s more powerful, commercial writing algorithms in July 2020, the writing got still more impressive. “The sheer jump in creativity and storytelling ability was heavenly,” says one veteran player. The system got noticeably more creative in its ability to explore sexually explicit themes, too, this person says. For a time last year players noticed Latitude experimenting with a filter that automatically replaced occurrences of the word “rape” with “respect” but the feature was dropped.
The veteran player was among the AI Dungeon aficionados who embraced the game as an AI-enhanced writing tool to explore adult themes, including in a dedicated writing group. Unwanted suggestions from the algorithm could be removed from a story to steer it in a different direction; the results weren’t posted publicly unless a person chose to share them.
Latitude declined to share figures on how many adventures contained sexual content. OpenAI’s website says AI Dungeon attracts more than 20,000 players each day.
An AI Dungeon player who posted last week about a security flaw that made every story generated in the game publicly accessible says he downloaded several hundred thousand adventures created during four days in April. He analyzed a sample of 188,000 of them, and found 31 percent contained words suggesting they were sexually explicit. That analysis and the security flaw, now fixed, added to anger from some players over Latitude’s new approach to moderating content.
Latitude now faces the challenge of winning back users’ trust while meeting OpenAI’s requirements for tighter control over its text generator. The startup now must use OpenAI’s filtering technology, an OpenAI spokesperson said.
How to responsibly deploy AI systems that have ingested large swaths of internet text, including some unsavory parts, has become a hot topic in AI research. Two prominent Google researchers were forced out of the company after managers objected to a paper arguing for caution with such technology.
The technology can be used in very constrained ways, such as in Google search where it helps parse the meaning of long queries. OpenAI helped AI Dungeon to launch an impressive, but fraught application that let people prompt the technology to unspool more or less whatever it could.
