Wednesday, January 18, 2023

More ChatGPT fails

Excerpts from text generated by the prompt “Write a 500-word analysis of.”

John Ashbery, “What Is Poetry”:

Ashbery writes that “poetry is a language that speaks to the soul,” suggesting that poetry is a way of expressing emotions and ideas that cannot be conveyed through more conventional forms of communication. He also writes that “poetry is the voice of the ages,” suggesting that poetry is a way of connecting with the past and with the collective human experience.
Elizabeth Bishop, “One Art”:
The theme of loss is further developed in the third stanza, where the speaker describes different types of losses such as “a you, or a night, or a home” which are all things that can be lost, but not necessarily things that can’t be replaced. The fourth stanza describes the loss of a “shifting, floating city” which is a metaphor for something that is not tangible and can’t be replaced. These examples are intended to show that the speaker understands that there are different types of losses, and that some are more difficult to accept than others.
Emily Dickinson, 764 [My Life had stood — a Loaded Gun]:
In the fourth stanza, the speaker describes how their power is “the engine of my heart” and how they are “the tongue of my power.” The use of the metaphor of the engine and the tongue suggests that the speaker’s power is something that drives them and gives them voice. This stanza also implies that the speaker’s power is something that is a part of their very being, and that it is something that they cannot separate from themselves.
Philip Larkin, “Annus Mirabilis”:
The third stanza of the poem is where Larkin’s cynicism becomes most apparent. He states that “the stairs are where the girls are,” suggesting that the sexual revolution was primarily focused on the physical act of intercourse rather than genuine emotional connection. He also notes that “the trash / Our young men collected / To build their heaven which exists / And cowers in them to this day.” This line suggests that the societal changes of the 1960s were superficial and that the individuals who were involved in the sexual revolution were ultimately unfulfilled.
William Butler Yeats, “The Wild Swans at Coole”:
Yeats writes that “Nineteen hundred years have come and gone / And I, a passer-by.” This line suggests that Yeats is aware that his own life is fleeting, and that he is just a temporary presence in the world. He also writes that “The years upon my back like some great beast” which symbolizes the weight of time on him.
In each case, ChatGPT produces phrases and lines nowhere to be found in the poem. My best efforts to find sources return nothing, though “Nineteen hundred years have come and gone / And I, a passer-by” sounds like a great start for an Ashbery poem, titled, perhaps, “A You, or a Night, or a Home.”

Yesterday I found ChatGPT producing a 500-word analysis of Ted Berrigan’s poem “Red Shift” that bore no relation to the poem and, again, contained phrases that don’t appear in the poem. I can’t imagine what it would be like to speak to a student who turned in this sort of stuff as genuine writing.

Related posts
A 100-word blog post generated by ChatGPT : I’m sorry too, ChatGPT : Spot the bot : Teachers and chatbots

comments: 7

Anonymous said...

next week i'm attending a webinar on writing a legal e-book using ChatGPT. looking forward to see how it works.

kirsten

Michael Leddy said...

Don’t tell Bryan Garner about this. : )

Sparky said...

A no shite Sherlock moment. I find the hysteria around these various AI models laughable. Yes, chatbots can now produce reasonable sounding prose but it's still word salad because that's how these things work. I use CNNs in my research, powerful stuff but you need to understand the limitations of this technology. Please feel free to properly edit this rant.

Fresca said...

This made me laugh. I know the AI has no personality, but it strikes me as a very naughty child, making lines of poetry up. Bad ‘bot! A you, a horse, a what???
Funny, but with worrisome implications.

Michael Leddy said...

@Sparky: I can’t edit comments, but even if I could, I wouldn’t. You're fine.

@Fresca: I wondered this afternoon if the fake lines are a matter of copyright concerns. And remember the bot also makes up (or at least did make up) facts: it had me winning the Ruth Lilly Poetry Prize, and Elaine touring the world as a pianist.

Anonymous said...

i just came across this which might be of interest: https://biblioracle.substack.com/p/chatgpt-cant-kill-anything-worth?sd=pf

i love the examples.

kirsten

Michael Leddy said...

Thanks, Kirsten.

Reading about what would score a 5 made me remember a conversation with a honors student who wrote an essay that argued x on one page and x’s opposite a page or two later. When I pointed out the contradiction, she didn’t seem fazed. “I get the idea,” I said, “that in high school it didn’t really matter what you said in an essay.” She agreed. It was just a matter of going through the motions — not unlike ChatGPT.