AI Dungeon, a silly text adventure generator, is perhaps the most well known application of GPT-2. (James Cao’20/Tech editor)
Back in February 2019, a research lab called OpenAI announced it had created a powerful machine learning, text-generating system called Generative Pre-trained Transformer-2 (GPT-2). Fearing of misuse, OpenAI didn’t release its full version to the public until November 2019.
GPT-2 generates text by predicting what the next word in a sentence should be based on raw data it is given and a starting statement. For example, if it is supplied with a large number of fake news articles and a headline, it can generate a cohesive story about fake news. The level of cohesiveness varies based on how much data it is given, but the main reason why it’s so impressive is because it’s universal. Most other machine learning systems are very specialized (for example, AlphaGo, the AI that beat the world champion Go player), while GPT-2 can generate anything text-related as long as it has the data it needs.
However, it is because of this flexibility that OpenAI made the initial decision not to release it to the public. It could be misused to create spambots, trolling machines, automatic racist commenters, propaganda, fake news, etc. At the same time, it could also be the path to true artificial general intelligence due to its versatility. After releasing some incomplete builds of GPT-2 to the public to see how it would be utilized, OpenAI decided to release it in full, keeping an eye out for any misuse.
With GPT-2 out in the open, people have been experimenting with the algorithm, and of all the experiments made, perhaps the most well-known one is AI Dungeon, a text-based story creator that uses stories from ChooseYourStory.com as raw data to create a fantasy, mystery, apocalyptic, zombie, or a custom-based scenario with which you can interact.
When players start their adventure, they are given a prompt describing the setting, their character, and other related pieces of info that helps set up the story. They can then input a response, and the system will continue to generate the story based off of that response and previous parts of the adventure. The back and forth between the AI Dungeon’s story and the player’s response continues until the player’s made-up character dies or completes a certain objective.
AI Dungeon’s rise to fame, however, was not a result of immersive stories, but rather because of the absolute randomness and hilarity of the stories. The player could go from recruiting knights to slaying an evil dragon to hiring another dragon with gummy bears to assassinate a king.
Although AI Dungeon is a comic application of GPT-2 and is by no means perfect, it’s still an excellent demonstration of how flexible GPT-2 is.
Other than creating fantasy scenarios, it has numerous other possible applications, such as machine translating, text summarizing, and even predicting the future.
