You Ask, I Answer: Can AI Truly Be Creative?

In today’s episode, we’re tackling a fascinating question: Can AI be truly creative? You’ll delve into the intricacies of human creativity, exploring how our brains generate ideas and innovate. Discover the striking similarities between human and AI creative processes, and understand the key differences that set them apart. Join the discussion as we explore the potential and limitations of AI in the realm of creative thinking.

You Ask, I Answer: Can AI Truly Be Creative?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

— START OF FILE you-ask-i-answer-ai-truly-creative.mp4.wav.txt —

Christopher Penn: In today’s episode, Paul asked the question, “Can large language models come up with new ideas and innovate? If we’re talking about divergent creative thinking, brainstorming, and stuff, then here’s my question: How does a human innovate? How do humans come up with new ideas?”

This is an important question. We should first know how humans create before we can answer about whether machines are capable of it. There are no shortage of academic papers online that talk about creativity in humans and how it works in the brain. So I put—took 30 of those papers on things like MRI brain scans of people who are doing creative tasks. I put them into a language model called NotebookLM, which is a free service by Google. And I asked the model, “How do humans think creatively?”

So here’s the short version. In the right-hand corner—there are a lot of people who are doing creative tasks, and I put them into a language model called NotebookLM, which is a free service by Google. And I asked the model, “How do humans think creatively?” So here’s the—in the right-hand corner, there are a lot of people who are doing creative tasks. And I put them into a language model called NotebookLM, which is a free service by Google. And I asked the model, “How do humans think creatively?”

So here’s the brain. In the right prefrontal cortex, which is somewhere over here, is where ideas start, right—where, and also this is where memory encoding tends to operate. Ideas are filtered then into the left dorsal—dorsolateral prefrontal cortex is over here, which is where we apply constraints, learned rules like language and short-term memory. So, the idea comes—starts here, and then it comes over here, and we start modifying the idea and saying, “Okay, let’s toss some things out.”

Ideas then go to the medial prefrontal cortex, which is somewhere over here, and then we start modifying the idea and saying, “Okay, let’s toss—” to retrieve long-term memories from long-term memory and make associations with the ideas. From here to here to here, you’re bouncing around, and then ideas go through this iterative loop in your brain, getting more idea variations from over here, applying more or less rules over here, and then augmenting and building on the ideas in here.

And all this is happening in the neural network that is our literal brains. Our brains are neural networks. You will hear that term in computing and in AI, but the OG neural network is between your ears.

And we don’t even understand how memory works, not really. We know where it’s stored, but we’re still very unfamiliar with just how memory works. But that’s the general process that can be observed in humans.

That should sound kind of familiar. How does a language model generate ideas? Well, yeah, first you prompt it and say, “Hey, I want to brainstorm something or other.” That prompt is broken down into—into tokens, and then embedded, and then that is sent by a portion of a language model called “retrieval heads” and bounces into the long-term memory of the model. And it activates those memories—the model’s memories—kind of attached to this prompt, and then all that stuff gets sent to a decoder.

The decoder takes the embeddings—the mathematical associations—and reassembles them back into words, concepts, and ideas. Then it applies rules based on the model’s environment, like things you’re not allowed to say, and then that’s dumped into short-term memory in the conversation.

It’s very similar conceptually to how humans do creativity. It is not that far off of, “Hey, I’ve got this thing. What things are like this? Let me come up with things that are like this,” right?

If I ask you to be creative about solving a problem like, “How do we—how do we get people to stop speeding?” or “How do we get people to eat healthier foods?” and you brainstorm, you are bouncing around ideas looking for other things to latch onto in your neural network. And you return those ideas, either by writing them down or speaking them.

Machines are very similar. Now there are differences. So, there’s differences in the ways that machines work. Machines, for example, have way more long-term memory than—than you or I do. A model like today’s Google Gemini 1.5, or Claude Sonnet 3.5, or GPT-4 Omni—these things have read every book available that’s online and open to the public. Think about it. It has been a long time since I’ve read this. I can’t remember hardly anything in here. I can remember like, “To be, and not to be.” Right, that’s it. I can—I can recall accurately, like, 10 tokens, 15 words out of 800,000. If you were to ask a language model, “Recite part of Act Three, Scene One from A Midsummer Night’s Dream,” the model could do it with way more accuracy than—than you or I can.

So, their memory is substantially better, and they have much more of it.

Think about something like Project Gutenberg, the open-source, free project to digitize and make freely available public domain books. AI has read all of it. You and I probably read just a fraction of it. So a model’s understanding of the world from an academic perspective is much more vast than you or I.

Second thing is machines don’t have selective memory beyond what we provide in the prompt. Our memory association—yours and mine—particularly for long-term memory, are rooted in emotions. We don’t store data factually, we store data in emotions. And when we—it’s funny, when we recall memories—when it actually brings data out of storage—we are regenerating it for the first time, based on what we remember. And what we remember is rooted in emotion.

Machines don’t have that. Machines don’t have emotions. So their—their recall mechanism is very different.

Think about this—this to be true. Think about—think about that first date with that special someone, right? Or think about that—that—that dinner you had that—that amazing, special occasion, the one you just—you can still—you can remember what it was. You can probably, mentally, recall the taste of it, like how—how well the thing was cooked and what sauce was on the side and what the dessert was, but you can’t remember what you had for lunch two weeks ago, because what you had two weeks ago was not emotionally resonant. Your brain didn’t reach the activation levels. It says, “Yeah, let’s store this data.” Like, “Yeah, it’s fine, whatever it was. It was fine.” Unless you got food poisoning. That wasn’t fine.

That’s how much—that’s how our human brains work. Machines do not work that way. Our memory is much more selective and as rooted in emotion.

Third, machines don’t have sensory information. They have limited amounts of vision, they have a lot of understanding of text, they don’t have auditory, not really. And they don’t understand—they don’t perceive the world the same way we do. Those perceptions change our associations, right?

If you—if I give you a sniff of a perfume, and that perfume happens to have an association with that special someone, past or present, suddenly, all these other memories activate in your neural network, and you’re like, “Oh, yeah. I remember that, and I remember this thing, and I remember that dance we went to. I remember this—” the—that evening in the park, and a lot of data gets invoked from that sensory information.

Machines have none of that. Machines, because they don’t have that data, they’re understanding the world is much narrower. It’s much more literal than what we have, which is much more sensory in nature. So, our brainstorming will be different, but our brainstorming will not be better or worse than machines. It’s just different.

And that’s what we conclude about AI and its ability to create—to do creative thinking. If we’re wondering which is better—human or machine for creative thinking—it depends on the task and the observed data.

Take something that is very heavily a language task like corporate strategy. Can AI brainstorm corporate strategy? Sure, absolutely. And it’s going to do that way better than any human being because as long as your prompts are good, that’s not a sensory task. That is an academic task, as a knowledge task, and it has read every strategy book on—on—on corporate strategy ever written. So it has more knowledge and more recall. It will crush that task. It will blow way past us.

Can AI brainstorm a—a new taste and net new taste combination for a dish? Yeah, but not really. Because that’s a high-sensory task. If you’re trying something that’s net new, a machine can’t taste what you’ve created. It can infer based on its knowledge of ingredients and cookbooks and all this stuff, but it doesn’t have the ability to stick its finger in and go, “Nope, needs more salt.” That sensory data doesn’t exist, and so a machine today cannot brainstorm and do creative thinking on high-sensory stuff, high-emotion stuff in the same way that you and I can. It’s just not possible today.

But that is changing. As robots go out into the world and start collecting sensory data and interact with the world, there’ll be that much more data for models to learn, and it will improve their ability to understand the world significantly better.

So, really good question. Thanks for asking. Talk to you on the next one.

If you enjoyed this video, please hit the like button, subscribe to my channel if you haven’t already. And if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.
— END OF FILE you-ask-i-answer-ai-truly-creative.mp4.wav.txt —

You Ask, I Answer: Can AI Truly Be Creative?

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest