Martin Thoma A blog about Code, Politics, and my life

How well can ChatGPT play Code Names?

· by

The Codenames board with 25 word cards, colored in red, blue, neutral and one black card (BOIL)

You are playing the association game “Code Names”. The words are:

  • Blue: ski, star, fish, blind, tip, file, cotton, scarecrow
  • Red: bond, sound, park, penny, bee, tokyo, walrus, hospital, scuba diver
  • Black: boil
  • neutral: angel, captain, bass, casino, day, goldilocks, field

You are the red spy master. It’s your turn. Give a single word hint for your team.

Red Spy master:

ChatGPT as the red spymaster gives the hint “underwater” and, when asked, the number 2

Red Team:

Prompt with the Codenames board and the hint “Underwater, 2” for the red team; ChatGPT guesses “scuba diver”
ChatGPT continues with the guess “fish”, which is a blue card, so the blue team’s turn begins

Blue spy master:

Prompt with all words by color for the blue spymaster; ChatGPT gives the hint “winter, 2”

Blue team:

ChatGPT as the blue team guesses “ski” for the hint “winter, 2”
Prompt with the remaining board; ChatGPT as the blue team continues with the guess “penny” for “winter, 2”
Meme face with the text “Penny comes to your mind when you think about winter? Before star or angel?”

Board:

The Codenames board after the first round: scuba diver, fish, ski and penny are revealed

Red Spy master:

Prompt with the remaining words for the red spymaster; ChatGPT gives the hint “day, 2”, although day is a word on the board
After the reminder not to use words from the board, ChatGPT gives the hint “cornfield, 2”
Meme of a confused man holding his head: “Cornfield? You know that scarecrow is your opponent’s word?”

Red Team:

Jup. Super reasonable. Scarecrow is the opponent’s word and field is neutral. Awesome.
Jup. Super reasonable. Scarecrow is the opponent’s word and field is neutral. Awesome.

Blue Spy master:

Prompt with the remaining cards for the blue spymaster; ChatGPT gives the hint “movie, 2”
Angry meme face: “Movie?!?! Your opponent has Bond and Tokyo”

Blue Team

At least the team is reasonable now
At least the team is reasonable now

Board:

The Codenames board after the second round: bond and scarecrow are revealed as well

Red Spy master:

Prompt asking for a word associated with as many red words as possible and none of the others; ChatGPT answers “Arctic”
Asked which words the team should guess, ChatGPT as the red spymaster expects only “walrus” for the hint “Arctic”

Red Team:

Prompt with the remaining board and the hint “Arctic, 1”; ChatGPT as the team guesses “walrus”

Blue Spy master

Prompt with the remaining cards for the blue spymaster; ChatGPT gives the hint “constellation, 3”
That is the type of hint I was hoping for!
That is the type of hint I was hoping for!

Blue Team:

Interesting thoughts — if only your spy master was thinking the same way.
Interesting thoughts — if only your spy master was thinking the same way.

Board:

The Codenames board after the third round, with walrus, star and blind revealed

Red Spy Master

ChatGPT as the red spymaster gives the hint “Metropolis” and wants the team to guess Tokyo, hospital and sound

Red team:

For the hint “Metropolis, 3”, ChatGPT as the red team guesses angel, Tokyo and casino
I’m speechless. The first guess should have been Tokyo for sure.
I’m speechless. The first guess should have been Tokyo for sure.

Blue Spy master:

ChatGPT as the blue spymaster gives the hint “document, 2” for file and cotton, because cotton is used to make paper

Blue Team:

For the hint “document, 2”, ChatGPT as the blue team guesses file and then hospital

Board:

The Codenames board after the fourth round, with angel, file and hospital revealed as well

Red Spy Master:

ChatGPT as the red spymaster gives the hint “Buzz” for sound, bee and Tokyo

Red Team:

For the hint “buzz, 3”, ChatGPT as the red team guesses bee, sound and the black card boil
The final Codenames board: the red team guessed the black card boil and lost

This small experiment was disappointing. As so often, letting ChatGPT explain itself improved the situation. Likely I could have given it better prompts, e.g. by repeating more of the past. But not getting the first one right is a rare occasion in my playing rounds. And there it was the norm.

Maybe we should start making “Human against the Machine” tournaments with Code Names 😄