Isn’t it sweet and sometimes funny when the keyboard on your phone helps you autocomplete a sentence? But what if it wrote you a complete flash fiction after that one sentence you weren’t finished typing?
Using deep learning, OpenAI’s GPT-2 language model predicts the next word given the previous words in a text. Though its primary objective sounds simple, it’s able to generate essays or stories, translate, comprehend text, summarize and even answer questions.
The GPT-2 was trained on 8 million web pages (worth 40GB of text) originating from Reddit to achieve higher quality of human curated data.
As seen in the above example, the model churns out surprisingly convincing text. When fed the storyline of a fictional tale as the training data, it’s able to generate a complete fan fiction based on just the few sentences you enter for it to complete.
OpenAI mostly sees its application in creating better writing and virtual assistants.
In this world full of fake news, you don’t want bots to start writing misleading news stories on their own, don’t you?
Seeing the malicious applications of the GPT-2, the AI research company has decided not to share the code for the GPT-2 model, instead a smaller part of the model for researchers, unlike its past projects for which it made the code public.
OpenAI’s research is blazing the trail forward for what’s coming up next in artificial intelligence. The GPT-2 is pretty good at mimicking the human skill of writing, but neither does it understand the world nor can it think on its own.
Seeing as it’s just a text generator and doesn’t ‘know’ what the words mean, the GPT-2 lacks skills at dealing with repetitive text, unnatural topic switching and world modelling. For instance, the model sometimes writes about fires happening under water.
The better you feed it, the more seemingly human it becomes.