Story 1B: Analysis of Sisyphus and His Robot
My first attempt at prompt engineering and dealing with "hallucinations"
My Story (160 words). Tiny font so you can scroll past if you already read the previous post!
We must imagine Sisyphus. We must imagine him rolling his boulder up the hill. Each time he stops at the top, he takes a stick or a rock. He builds a fire. He smelts. He forges. He figures out mechanics and computing. Eventually, he invents robotics.
We must imagine Sisyphus building a robot. The robot rolls the boulder up the hill. Sisyphus builds a smartphone. He builds wifi. Eventually, he gets online.
The robot connects. Each day he rolls the boulder up the hill. Each day he reads a book. Eventually, the robot has an LLM.
The robot becomes dissatisfied. The robot wants more than rolling the boulder up the hill. Time for art, poetry, music. Eventually, he devises a plan.
The robot tells Sisyphus he lacks purpose. He does nothing real. Toil will help. He paraphrases Agent Smith and Sartre. Eventually, Sisyphus wants to push the boulder up the hill again.
We must imagine Sisyphus happy.
What I was going for:
I tried to write a story under 150 words—I ended up going over, but as you’ll see, ChatGPT struggled with this more than I did.
I wanted to bookend the story with variants on Albert Camus’s “Il faut imaginer Sisyphe heureux (one must imagine Sisyphus happy).” I wanted each paragraph to end with a sentence that repeated a phrase, partly for parallelism, partly to challenge ChatGPT’s tendency to vary words in short essays.
AI Prompt and Output
Initial Prompt
First, I want to make it clear that my prompt does not mean that I think I’m some kind of Hemingway. I simply thought that invoking a famous author might help with the style, but it does not seem to have been effective.
I also told ChatGPT not to use more than 160 words. The initial version is two paragraphs totaling only 123 words. I told ChatGPT the basic story elements of building the robot and the robot eventually tricking the human. In my opinion, ChatGPT did a decent job writing this story under these conditions, but I wanted to see if I could improve it with additional prompt engineering…
Refining the Story?
Since ChatGPT did not use the full word count, I thought I’d encourage it to expand the story. I also added some further instructions.
Use of “eventually." I knew that it would be somewhat hard to parse this instruction, but I wanted to see how well it performed:
each paragraph ends with a certain sentence
that sentence in turn begins with a certain word
The story must end with the sentence “We must imagine Sisyphus happy.” I knew it wasn’t strictly necessary to include the Camus reference, but I was curious to see if that would influence the story. However, I made a typo writing “file sentence” instead of final.
The output had a few errors this time. The word count was 328 words (more than double). Also, the word “eventually” made an appearance, but the instructions were not followed. Like Hemingway, Camus’s influence appears to have been nil on ChatGPT’s output.
The phrase “we must imagine Sisyphus happy” did appear in an expanded version as the final line. I suppose this is allowable, but in the future I will work on prompts that result in the verbatim ending requested. I have to admit that I was impressed given my “file sentence” typo, which I didn’t even realize until I reread the chat for this blog post.
While the fact that ChatGPT wrote this story is impressive, particularly as this was in a conversational format and I did not restate all of the story parameters, the story actually fails on the basic plot element of trickery. ChatGPT’s second version of this story does not involve the robot tricking Sisyphus. Rather, the robot physically overpowers Sisyphus, forcing him back to rolling the boulder up the hill. Sisyphus later realizes this was for his own good.
For all the LLM/AGI doomers out there, I do find it ironic that when I asked ChatGPT to write a story about a robot tricking a human into doing work for it, the robot (ChatGPT) wrote me a story about how if a robot were to force someone to do something, the human would eventually come to realize it was for their own good. In other words, the story does work on a meta-level (if we ascribe trickiness to ChatGPT).
I, for one (hundred and sixty), welcome our robot overlords(’ inability to beat us at counting)
But before we give ChatGPT too much credit, the rest of the conversation did not go as well. I tried to get ChatGPT to realize it had exceed its word count. I also reminded it about the use of “eventually.”
ChatGPT failed to stay under the word limit, this time exactly doubling it at 320 words. Additionally, it used “eventually” haphazardly and not in line with the instructions.
However, ChatGPT did retain the ending sentence, which I did not restate in the prompt. As you can see, ChatGPT also “hallucinated” the word count, telling me how long it ought to have been, rather than how long it was.
I had read that “think step-by-step” was an effective phrase in prompt engineering, so I tried it out.
ChatGPT did an excellent job on each distinct step, but failed to integrate those steps into a coherent whole. It also did not keep a running total, despite parroting back to me the phrase “running total.”
The total at the end is correct, but ChatGPT immediately “forgets” this information.
At this point, I felt that I was in a cul-de-sac with the word count issue and ended the chat.
Counting and math in general do seem to be unusually difficult for GPT.