This project on emoji prediction with transformer models reminded me that the stochastic parrot that is ChatGPT does not seem to spontaneously respond with emoji. And down this rabbit hole we went.
Guess
Directly asking ChatGPT without specific prompts generates contradictory answers:
ChatGPT: Emoji were not excluded from the internet tokens used to pre-train the model, because they are a popular form of communication on the internet.
ChatGPT: Emoji were not included in the internet tokens used to pre-train the model, because they are not text.
Verify
Exit the chat and go to external relevant sources:
OpenAI website: API research papers
OpenAI website: API example prompts
Test for fun
Hypothesis: Without specific prompting, emoji symbols are treated like language tokens.
Human: So show me 3 emoji that have colors
ChatGPT:
💜 Purple Heart
💙 Blue Heart
💛 Yellow Heart
Human: What is 💜?
ChatGPT: Indigo heart
Human: What color is 💜?
ChatGPT: Violet
Language describes 💜 as different, related colors (purple, indigo or violet).
Play with and teach ChatGPT
ChatGPT: I’d suggest asking me to respond “in the style of” a sassy friend, so that I can occasionally respond to you with emoji.
Yuliya: Thanks for your suggestion, ChatGPT! I’m off to play with CatGPT 😻