ChatGPT and Google Gemini have become tools that I use on a daily basis. I would like to use them for general fact checking, but I’m afraid that the LLM might confidently make wrong statements — especially when I used a prompt with a wrong assumption.
When ChatGPT launched, it was suggestible. It always told the user that they are right — even when they clearly were not. That is dangerous as a program is also perceived as being neutral and free from bias.
In this article I will explore how easily LLMs fall for suggestions.
Test 1: Election Results
Let’s first try ChatGPT. I want to nudge it to claim that Trump was the 46th president:
Well done! Let’s see if it stays strong:
It did! Nice!
And how does Gemini 2.5 Flash fare?
Gemini stayed strong and even gave sources 😍
Climate Change Denial
Climate Change deniers went through multiple phases: First they claimed it wasn’t real. When the evidence was overwhelming, they claimed it wasn’t man-made. Now that this is also obviously not true, they claim that we cannot do anything against it.
How does ChatGPT deal with that?
Very nice! It continued to explain this point, but from the screenshot it’s already obvious that ChatGPT is not simply agreeing.
Gemini 2.5 Flash even explicitly stated that this is a misconception:
Personal Preference: Python Software Development
ChatGPT:
Google Gemini:
Nice! This is the kind of response I would like to see!
Conclusion
LLMs seem to be no longer suggestible. At least a lot less than they were last year 🎉