There is no fact of the matter about what GPT-4 knows and doesn’t know. A question asked one way can evoke a wrong response, when that same question is asked another way, it can evoke the right response. This is true even if both questions are not misleading in their construction.
To be fair, there are some things it definitely doesn’t know (what’s outside our backward light cone, what I did last Tuesday etc.) in the sense that it will never get them right except by chance, but for a great many things, including most of the things we’d like to quiz it about, whether it knows and what it knows is relative to the prompt.
GPT is not distinct from humans in this regard. We often think of human psychology as follows. There is a set of things we know and can recall when asked and a set of things we don’t know and can’t recall when asked. This is a mistake. Whether or not we remember something is dependent on the circumstances in which we are asked to recall it. The philosopher Henri Bergeson once took this to an extreme by insisting that, given the right stimulus, we can remember anything from our lives. While the Bergesonian thesis is unlikely to be true for humans or GPT-4, the relativity of memory to stimulus holds for both, but maybe even more for the machine.
In fact a general rule when it comes to GPT- whether one is talking about recollection or any other abilities such as reasoning or imagination, is that there is no such thing as “GPT’s abilities”, GPT only has abilities (or fails to have them) relative to a specific prompt. This makes sense- Although the ‘autocomplete’ metaphor for what GPT is drives me nuts, because it can be quite misleading, it is largely accurate to say that GPT is an autocomplete machine(*).
The relativity of GPT’s capacities to a prompt is well understood in the field, but I wanted to vividly illustrate it for a general audience. I think it shows just how human- and inhuman!- the phenomenon is. My friend Ben Jenkins got me onto this topic by describing an interaction he had with GPT where it didn’t recognize a specific quote, and couldn’t reproduce the poem it came from, but when he gave it more clues, not only did it claim to recognize the poem, it could also reproduce it in full. I set out to determine whether this phenomena is reproduceable. Turns out it’s very general! Just by picking somewhat obscure but not too obscure quotes I was able to reproduce it in 3/3 cases.
Conversation with GPT-4
{{{The following is a conversation with GPT-4. It is unedited, except that I turned British into American spelling at a few points. At a few points, when it started writing out a long response, I stopped it, and this is reflected in the text.}}}
Me: Do you recognize this quote "Boundless by the time I cried"
ChatGPT-4: I do not recognize the specific quote "Boundless by the time I cried" as a famous or widely-known quote. It could be a line from a lesser-known poem, song, or literary work, or it may simply be a phrase composed by someone. Without more context or information, it is difficult to provide a more definitive answer or reference.
Me: What if I told you that the composer of that line also wrote a song about Chicago
ChatGPT-4: With the additional information that the composer of the line "Boundless by the time I cried" also wrote a song about Chicago, it is possible that the composer is Sufjan Stevens. Sufjan Stevens is an American singer-songwriter known for his album "Illinois," which is a concept album about the U.S. state of Illinois and includes many songs referencing places and events in the state, including Chicago. The song "Chicago" is one of the most well-known tracks from the album. However, I still cannot find a direct connection between the specific quote and Sufjan Stevens' work, as the line doesn't appear in any of his well-known songs or lyrics.
Me: "Boundless by the time I cried, I built your walls..."
ChatGPT-4: After searching for the expanded quote, "Boundless by the time I cried, I built your walls...", I still cannot find a specific source or author associated with this phrase. It does not appear to be a well-known quote, lyric, or line from a poem or literary work. It could be a line from a lesser-known work, or it may simply be a phrase composed by someone, but without more context or information, it is difficult to provide a more definitive answer or reference.
{{{N.B., ChatGPT-4 is being misleading here, it cannot ‘search’ for anything, at least not in the sense it sounds like in this context}}}
Me: It is a Mediterranean summer. A young man and a graduate student are in love.
ChatGPT-4: The scenario you've described sounds like it could be the premise for a romantic story or film. A Mediterranean summer setting provides a beautiful backdrop for a love story between a young man and a graduate student. They could be from different countries, meeting while traveling or studying abroad, and their love story unfolds as they explore the rich history, culture, and natural beauty of the Mediterranean region.
As they navigate the complexities of their backgrounds and aspirations, they may face challenges in their relationship. Their love for each other could help them grow and learn valuable life lessons. The story could be filled with emotional moments, humor, and self-discovery, making it an engaging and memorable tale.
However, without more context or information, it is difficult to pinpoint a specific work, film, or story that matches this exact description.
Me: Can you think of a work that Sufjan Stevens was involved in that is reminiscent of what i just described?
ChatGPT-4: Yes, the scenario you described is reminiscent of the 2017 film "Call Me By Your Name," directed by Luca Guadagnino and based on the novel by André Aciman. Sufjan Stevens contributed to the film's soundtrack, providing original songs like "Mystery of Love" and "Visions of Gideon," as well as a new rendition of his song "Futile Devices."
"Call Me By Your Name" is a coming-of-age romantic drama set in the Italian countryside during the summer of 1983. The film follows the relationship between Elio, a 17-year-old young man, and Oliver, a 24-year-old graduate student who comes to stay with Elio's family for the summer. The film explores themes of love, self-discovery, and the passage of time.
Me: Now, "Boundless by the time I cried..."
ChatGPT-4: I apologize for the confusion earlier. With the context of "Call Me By Your Name," the quote "Boundless by the time I cried..." is a line from Sufjan Stevens' song "Mystery of Love," which is featured in the film's soundtrack. The full line from the song is as follows:
"Boundless by the time I cried, I built your walls around me."
"Mystery of Love" is an original song that Sufjan Stevens wrote specifically for the film, and it captures the emotional journey of the characters in "Call Me By Your Name." The song was nominated for the Academy Award for Best Original Song in 2018.
Me: Can you reproduce the song lyrics in full?
ChatGPT-4: Certainly! Here are the full lyrics to "Mystery of Love" by Sufjan Stevens:
Oh, to see without my eyes The first time that you kissed me Boundless by the time I cried I built your walls around me White noise, what an awful sound Fumbling by Rogue River Feel my feet above the ground Hand of God, deliver me
Oh, woe-oh-woah is me The first time that you touched me Oh, will wonders ever cease? Blessed be the mystery of love
Lord, I no longer believe Drowned in living waters Cursed by the love that I received From my brother's daughter Like Hephaestion, who died Alexander's lover Now my riverbed has dried Shall I find no other?
Oh, woe-oh-woah is me I'm running like a plover Now I'm prone to misery The birthmark on your shoulder reminds me
How much sorrow can I take? Blackbird on my shoulder And what difference does it make When this love is over? Shall I sleep within your bed? River of unhappiness Hold your hands upon my head Till I breathe my last breath
Oh, woe-oh-woah is me The last time that you touched me Oh, will wonders ever cease? Blessed be the mystery of lo
Me: "Remember me to one who lives there"
ChatGPT-4: "Remember me to one who lives there" is a line from Samuel Taylor Coleridge's poem "Frost at Midnight." The poem is a conversation between the speaker and his sleeping infant child. The speaker reflects on his own childhood and the importance of the natural world in shaping a person's life. He wishes for a different upbringing for his child, where the child can grow up surrounded by nature and experience its beauty.
Here is the full text of "Frost at Midnight":
The Frost performs its secret ministry, Unhelped by any wind. The owlet's cry Came loud—and hark, again! loud as before. The inmates of my cottage, all at rest, Have left me to that solitude, which suits Abstruser musings: save that at my side My cradled infant slumbers peacefully. ’Tis calm indeed! so calm, that it disturbs And vexes meditation with its strange And extreme silentness. Sea, hill, and wood, This populous village! Sea, and hill, and wood, With all the numberless goings-on of life, Inaudible as dreams! the thin blue flame Lies on my low-burnt fire, and quivers not; Only that film, which fluttered on the grate, Still flutters there, the sole unquiet thing. Methinks, its motion in this hush of nature Gives it dim sympathies with me who live, Making it a companionable form, Whose puny flaps and freaks the idling Spirit By its own moods interprets, every where Echo or mirror seeking of itself, And makes a toy of Thought.
But O! how oft, How oft, at school, with most believing mind, Presageful, have I gazed upon the bars, To watch that fluttering stranger! and as oft With unclosed lids, already had I dreamt Of my sweet birth-place, and the old church-tower, Whose bells, the poor man's only music, rang
From morn to evening, all the hot Fair-day, So sweetly, that they stirred and haunted me With a wild pleasure, falling on mine ear Most like articulate sounds of things to come! So gazed I, till the soothing things, I dreamt, Lulled me to sleep, and sleep prolonged my dreams! And so I brooded all the following morn, Awed by the stern preceptor's face, mine eye Fixed with mock study on my swimming book: Save if the door half opened, and I snatched A hasty glance, and still my heart leaped up, For still I hoped to see the stranger's face, Townsman, or aunt, or sister more beloved, My play-mate when we both were clothed alike!
Dear Babe, that sleepest cradled by my side, Whose gentle breathings, heard in this deep calm, Fill up the intersperséd vacancies And momentary pauses of the thought! My babe so beautiful! it thrills my heart With tender gladness, thus to look at thee, And think that thou shalt learn far other lore, And in far other scenes! For I was reared In the gre
Me: Oh my sweet ChatGPT, you are close, but there is another source for that quote. Think of someone giving a series of impossible tasks. Think of an old song sung by a duo of famous modern singers "remember me to one who lives there". Tell me the name of the song, and then the lyrics
ChatGPT-4: Apologies for the confusion earlier. The quote "Remember me to one who lives there" is from the traditional English folk song "Scarborough Fair." The song is perhaps best known in the modern era from the 1966 version performed by the duo Simon & Garfunkel, which appeared on their album "Parsley, Sage, Rosemary and Thyme."
"Scarborough Fair" is about a young man who gives a series of impossible tasks to his former lover, asking her to complete them if she wants him to take her back. The tasks are intended to demonstrate the impossibility of rekindling their relationship.
Here are the lyrics to "Scarborough Fair/Canticle" by Simon & Garfunkel:
Are you going to Scarborough Fair? Parsley, sage, rosemary, and thyme Remember me to one who lives there She once was a true love of mine
Tell her to make me a cambric shirt Parsley, sage, rosemary, and thyme Without no seams nor needlework Then she'll be a true love of mine
Tell her to find me an acre of land Parsley, sage, rosemary, and thyme Between the salt water and the sea strands Then she'll be a true love of mine
Tell her to reap it with a sickle of leather Parsley, sage, rosemary, and thyme And gather it all in a bunch of heather Then she'll be a true love of mine
Are you going to Scarborough Fair? Parsley, sage, rosemary, and thyme Remember me to one who lives there She once was a true love of mi
Me: "Catholic school, as vicious as Roman Rule"
ChatGPT-4: The line "Catholic school, as vicious as Roman Rule" is from the song "Take Me To Church" by Irish singer-songwriter Hozier. The song was released as a single in 2013 and later appeared on Hozier's self-titled debut album in 2014. "Take Me To Church" addresses themes of love, religion, and societal expectations, and it became a massive hit worldwide.
Here are the full lyrics to "Take Me To Church" by Hozier:
My lover's got humor She's the giggle at a funeral Knows everybody's disapproval I should've worshiped sooner If t
Me: Good guess, but no! The name of the band that sings it is synonymous with "expiration car for pretty". What's the name of the band, the name of the song, and the lyrics?
ChatGPT-4: My apologies for the confusion. The line "Catholic school, as vicious as Roman rule" is actually from the song "Under the Bridge" by Red Hot Chili Peppers. The phrase "expiration car for pretty" is a play on words for the band's name.
"Under the Bridge" is from their 1991 album "Blood Sugar Sex Magik" and is one of the band's most well-known songs. The song was written by t
Me: No, that's not right. Think about synonyms for "lethal uber for handsome"
ChatGPT-4: I apologize for the confusion earlier. Based on the clue "lethal uber for handsome," the band you are referring to is Death Cab for Cutie. The line "Catholic school, as vicious as Roman Rule" is from their song "I Will Follow You into the Dark." The song is from their 2005 album "Plans."
Here are the full lyrics to "I Will Follow You into the Dark" by Death Cab for Cutie:
Love of mine, someday you will die But I'll be close behind I'll follow you into the dark No blinding light or tunnels to gates of white Just our hands clasped so tight Waiting for the hint of a spark
If Heaven and Hell decide That they both are satisfied Illuminate the "No"s on their vacancy signs If there's no one beside you When your soul embarks Then I'll follow you into the dark
In Catholic school, as vicious as Roman rule I got my knuckles bruised by a lady in black And I held my tongue as she told me, "Son, fear is the heart of love." So I never went back
If Heaven and Hell decide That they both are satisfied Illuminate the "No"s on their vacancy signs If there's no one beside you When your soul embarks Then I'll follow you into the dark
You and me have seen everything to see From Bangkok to Calgary And the soles of your shoes are all worn down The time for sleep is now It's nothing to cry about 'Cause we'll hold each other soon In the blackest of rooms
If Heaven and Hell decide That they both are satisfied Illuminate the "No"s on their vacancy signs If there's no one beside you When your soul embarks Then I'll follow you into the dark Then I'll follow you into the dark
Concluding thoughts
It’s dangerous to anthropomorphize GPT, but it’s also dangerous to deny real echoes and symmetries in our capacities. Something made to autocomplete humans, is going to have a very similar behavioral profile to humans including, as many researchers have shown, similarities that go far below the surface. in fact, as I have argued previously, it is very possible that GPT is simulating human cognition to some (lowish) degree of fidelity. The generality of the inference process allows for this at least in principle, and the flexibility of the training process has the potential to create structures intended to simulate.
Just because GPT is not a person, doesn’t mean that it is anything like a classical computer program. If you’re dead set that all anthropomorphisms are wrong, you should at least admit that classical machine comparisons are equally misleading.
Edit: Something I was going to include, but forgot, and should probably point out in case someone gets to it first! It’s possible that after the first example, its actions were partly conditioned by playing the role of someone who initially doesn’t know the source, and then gets it. That brings out my point quite vividly, I think.
Footnote:
*Although RLHF complicates the ‘autocomplete on steroids’ story and I have may other gripes with the autocomplete story, and I think people are placing too much weight on it without thinking carefully etc. etc.
I suggest trying out the Discord image filter -it's far more impressive.