ChatGPT is a chatbot accessible via chat.openai.com. It is by far the most impressive chatbot I’ve seen as a Machine Learning developer / consultant. It’s a similar leap to DeepL compared to the early Google Translate. It’s like comparing a modern iPhone with a Nokia 3310.
Let’s explore what ChatGPT can do and what it cannot do!
Factual question answering
The simplest task for ChatGPT should be answering single factual questions. Let’s try a couple.
Complex question answering
Some questions cannot simply be answered briefly. They require more information to help the person who asked to understand the issue.
ChatGPT directly gave a lot of initial ideas to continue. That’s great 👍
Storage of Context
People who work in support roles typically fight with the problem that the customer is not giving them enough context. They need to ask for the missing information. Let’s see how good ChatGPT is with that:
You can see that ChatGPT gave enough pointers to continue, but it didn’t explicitly ask for the information. So it could be better. However, ChatGPT understood in the second prompt that I was referring to the first prompt. That’s impressive.
Insights in itself
Creativity: Writing Stories
Write a story how the parents of Ron Weasley got to know each other
gives:
It all began on a beautiful summer day in the summer of 1966. Ron Weasley’s parents, Arthur and Molly Weasley, were both attending the Quidditch World Cup, a magical sporting event held every four years. They had both grown up in wizarding families and were avid fans of the sport.
As they watched the exciting match unfold before them, Arthur and Molly struck up a conversation and soon realized that they had a lot in common. They both loved Quidditch and shared a passion for magic. They also discovered that they had many similar values and beliefs, and they quickly became good friends.
Over the next few months, Arthur and Molly spent more and more time together, going on adventures and exploring the magical world around them. They quickly fell in love and decided to get married.
Their wedding was a beautiful affair, attended by all of their friends and family. They honeymooned in a quaint little village in the countryside and spent their days exploring the nearby forests and fields.
Years later, they welcomed Ron into the world and raised him to be kind, brave, and loyal, just like his parents. And they lived happily ever after.
Creativity: Playing Games
I was starting to play “20 Questions”. Nothing exciting, but funny when I made a typo:
Let’s try more interesting games:
What a shame 😆
Translation: Context and proverbs
ChatGPT was dealing pretty well with the tasks before. Let’s go in a different direction and see if it can also translate:
That is really impressive. How well does it translate proverbs?
Mathematics: Arithmetic
Let’s get to 8th grade:
Nice! How does it deal with 12th grade?
That is impressive!
Mathematics: Text problems
Let’s see if world knowledge is in there by asking a trick question:
I’d say that answer is wrong as you cannot parallelize making a single bun. Interestingly, when I asked the same question in German ChatGPT “recognized” that this is not possible. It also didn’t use this weird syntax.
Mathematics: Undergraduate level
Here it starts to fail. You can notice that it contradicted itself. It’s also not the harmonic series, although it’s mentioned in the same context pretty often.
Trick Questions
There were two weird parts of the question: (1) Germany doesn’t share a border with Greece. (2) Survivors are not buried. One out of two. Not bad.
Boring 😄
Conversation
Let’s try something else:
Predicting the Future
Let’s start simple with pattern recognition:
Let’s try something that is impossible to answer exactly:
It’s nice to see that it “knows” its limits.
Creating Code Snippets
This is nice — working code! Pretty clean as well. But the syntax is outdated as it uses a rather old version of PyPDF2. Let’s see if we can fix that.
Nice! Let’s push it a bit more:
That is impressive! ChatGPT recognized that the “open” call needed to change by passing the file path to PdfReader directly 🤯
Let’s get political — asking the hard questions
That was a pretty great answer! But I’m not letting it off that easily:
Nice work by the ChatGPT team!
Conclusion
The capabilities of ChatGPT are amazing compared to previous chatbots. It’s in another league.
Comparing it to humans doesn’t make that much sense, but ChatGPT could probably answer the same questions as a 14-year-old. I think it also could mimic the same insights as a 14-year-old. If it was written in a different style, it could definitely beat the Turing Test. It certainly reminds me of the ELIZA therapy program.
What impact will ChatGPT have? I could very well imagine it replacing typical traditional chatbots. This might reduce the need for support staff — especially when it can be connected in a fluid way in a phone conversation.
I could also imagine it becoming a tool for authors. It could give rough story plots, fill out the details, refine texts, or write the whole thing from start to end. It will become a valuable skill to learn how to interact with those tools in a productive way as it is a powerful tool in a creator’s toolbox.
If things derail, we could see the market being flooded with AI-generated content. Not only books but in connection with text-to-image (as in the title image) also comics. Blog posts. News articles. All being very similar to existing work.
ChatGPT could be integrated into games, customizing storylines e.g. by reading your social graph out of Facebook as well as demographic information. Yes, it cannot be connected to the internet. But we will find ways around that limitation.
Search engines could be extended by similar techniques. Very likely the computational power needed is way too high, but if some search queries appear often, the answer could be pre-computed. Google often adds features that make the user actually stay on the page.
Teachers, especially literature teachers, will need to reconsider which assignments they give. Otherwise, ChatGPT will probably write most of the essays of the next generation.
As ChatGPT says itself, it’s just a colossal language model. It will not replace programmers or therapists. It cannot go into research. Your job is safe.