Fireside Chat: Music and AI, Part 4: Teaching AI to Write Lyrics: A Deep Dive into Chamber Choir Music

In today’s episode, we roll up our sleeves and dive into the fascinating world of AI lyric generation! You’ll witness a step-by-step process of teaching AI to write lyrics for a chamber choir piece, a unique challenge even for the most advanced models. We’ll use Google’s AI tools to generate lyrics, refine them based on expert feedback, and even create a scoring rubric to evaluate their quality. Tune in to see how AI tackles this creative challenge and learn valuable insights into the evolving landscape of music composition with artificial intelligence.

Fireside Chat: Music and AI, Part 4: Teaching AI to Write Lyrics: Deep Dive into Chamber Choir Music

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

Christopher S. Penn – 00:01
In this five-part series on music and AI, filmed at the 2024 Marketing AI Conference, I sit down with composer Ruby King to discuss AI music and the future of creativity. This is part four, teaching AI to write lyrics — a deep dive into chamber choir music. Okay, so let’s do this. Let’s transition here. I’m going to go ahead and share my screen, and we’re going to see how well — yay, infinity tunnel — let’s go here to Bard first, and then AI Studio. So I’m going to use Google’s AI Studio, and the first thing I want to do is I want to get an understanding of what constitutes good songwriting. So I’m going to ask you to evaluate the model’s thing. So your topic is best practices for — what genre of music do you know very well that has lyrics? Pop, rock —

Christopher S. Penn – 00:58
— I guess it’s more grunge.

Ruby King – 01:00
No, no, it’s more — it’s classical pieces that have lyrics.

Christopher S. Penn – 01:08
I mean, Gregorian chants.

Ruby King – 01:11
No, chamber choir music or something like that, where there’s four parts or that kind of thing. I can do that.

Christopher S. Penn – 01:18
Lyrics to chamber choir pieces. So that is what we’re going to go for. We’re going to choose the Gemini model. We’re going to choose an output length of 16,384 tokens.

Ruby King – 01:32
Does it have to be with words for this one?

Christopher S. Penn – 01:34
Because you’re looking at — well, we wanted to do stuff with lyrics, and the reason we want to do it with lyrics is specifically because, for tools like Suno in particular, part of the generation is the lyrics themselves. That’s part of the prompt. So let’s see what Gemini has to say: “Best practices for generating lyrics for chamber choir pieces. Understanding the vocal capabilities and limitations of a chamber choir. Voice types and ranges. Blending and balance. Ensemble size and dynamics. Text selection. Text — choosing appropriate texts. Considerations for length, subject matter. Word painting and musical expression.”

Ruby King – 02:18
Example, word painting. That’s good.

Christopher S. Penn – 02:20
Okay. “Syllabic versus melismatic setting.”

Ruby King – 02:24
Yeah. Melismatic is when there was that singing before where it had no words.

Christopher S. Penn – 02:29
Okay, got it.

Ruby King – 02:31
AI doesn’t do that terribly well, but it’s singing multiple notes to a sound.

Christopher S. Penn – 02:35
Okay. “Creating unified, engaging musical structure, form, and development. Strophic through composed ABA. Vocal lines and harmony, repetition and variation. Practical considerations of performance. Score clarity and readability. Accessibility and singability. Common mistakes of less experienced practitioners. Writing ranges that are too wide or uncomfortable.”

Ruby King – 02:53
Yep, absolutely.

Christopher S. Penn – 02:55
“Neglecting vocal blended balance.”

Ruby King – 02:59
Can be a choice, but yeah.

Christopher S. Penn – 03:01
Okay. “Overlooking dynamics on ensemble size. Choosing texts that are unsuitable for choral music.”

Ruby King – 03:07
Yeah.

Christopher S. Penn – 03:08
“Overusing or misusing word painting. Inconsistent or inappropriate syllabic choices. Lack of musical structure development. Weak or unclear form. Uninstrung or unbalanced vocal lines. Neglecting practical considerations. Chamber choirs can — we sing quiet, introspective music that is generally to be true, but is actually false.” Really? “Rhyming lyrics are essential for chamber music.” Over there.

Ruby King – 03:34
I wouldn’t say so. A lot of the time, we’re singing just words straight from the Bible.

Christopher S. Penn – 03:38
Right. So they’re saying these are things that people believe to be true but are actually false.

Ruby King – 03:42
Okay. Yes, that’s true.

Christopher S. Penn – 03:43
“Latin texts are always more appropriate.”

Ruby King – 03:46
Yeah.

Christopher S. Penn – 03:47
Okay. “Things that are believed to be false but are actually true. Simple text can inspire profound choral music. Silence can be a powerful element. Collaborating with a living poet can enrich the creative process. Tips and tricks. Listen deeply to singers. Sing the parts yourselves. Study a choral score.” So these are all things you would say are appropriate for what you should know to create good chamber music.

Ruby King – 04:13
Yeah, I mean, I’ve not seen anything I disagree with.

Christopher S. Penn – 04:16
Okay. Is there anything missing that you know of that it didn’t know?

Ruby King – 04:23
Did it speak about the spaces that things are being sung in? I didn’t see that there. But it does also — if you’re singing in a really big church, you might choose — the amount of singers might be different with how many parts you split into.

Christopher S. Penn – 04:37
No, because we specified writing lyrics.

Ruby King – 04:41
Yes. Yeah. Okay. Yes. Lyrics also matter here, though, because if you’ve got — chamber choir is four parts, usually. So if you’ve got — if you’re writing lyrics and you’re in different spaces, you might want to write more words in a shorter period of time with lots of people doing different words. If you don’t care — what if — if you’re in a church service, usually when you’re hearing chamber choir, you want to hear the words that are being said because it is about worship, and that is the idea. If it’s a concert of chamber choir music for a different purpose, then you might have lots of different interweaving lines. But when it was for church services, I’d mostly have everyone singing the same word at the same time, but different notes.

Ruby King – 05:30
So it was nice to hear, but a few passing notes in emphasizing words, but it does change with the purpose of the service or the purpose of the music.

Christopher S. Penn – 05:42
Okay, so next, let’s have it create a scoring rubric. So next, create a scoring rubric that can evaluate a set of chamber choir lyrics and how well they adhere to our best practices and tips and tricks so far. The individual items in the scoring rubric should be scored as variable integers: zero to three, zero to five, zero to seven, etc. The rubric — hide that because I can’t see what I’m doing. There we go. The rubric as a whole should total 100 points. The purpose is to give lyrics an assessment of how well they conform to best practices. So let’s see what the model comes up with. So it’s going to say, “Here is how to score a set of lyrics.”

Ruby King – 06:55
Oh boy, okay. Just letting you know, this — again.

Christopher S. Penn – 07:05
Well, this is — this scoring rubric is one of the magic terms in generative AI, where you’re saying to the model, “I want you to evaluate.” And instead of saying, “Model, is this good or bad?” — because the model doesn’t know — you give it a set of scores that then it can use as a baseline. Say, “Here’s how — what I think I understand about this.” So let’s save this. This is our Gemini chamber choir lyrics gem rubric. Okay, so we’ve got our rubric now. Now let’s have it generate some lyrics, some chamber choir lyrics. What topic should we use?

Ruby King – 07:57
Usually, it’s sacred.

Christopher S. Penn – 07:58
Okay, let’s not offend anyone.

Ruby King – 08:00
Okay.

Christopher S. Penn – 08:04
Remember, this is a generative AI model, so it can make anything about anything.

Ruby King – 08:09
Okay. Chamber choirs are usually used for slow, solemn things. So if we do — usually, it’s quite effective if it’s sad or crying. So it just — you can have more upbeat ones, but usually, that’s all right.

Christopher S. Penn – 08:27
So let’s do this. Alright, next, generate two candidate sets of lyrics. The overall topic for the song should be a cat that is prowling around the streets of London, hungry. The lyrics should reflect this general situation and tell a story. Generate two candidate sets of chamber choir lyrics for this story. London’s tomcat prowl. Soprano one, alto one.

Ruby King – 09:16
Did it just start the word with “sh”? How are you meant to sing “sh”? I mean — okay, some of the time it does just write that, and then you make the sound, but we had music, though. We literally had to all go “quack” at the same time with notes. That’s just what happens, but it’s not a strong start.

Christopher S. Penn – 09:37
Okay, so we have candidate one, London tomcat prowl.

Ruby King – 09:43
Oh, they’ve got it in different — are they split into — okay. Soprano one and two. Okay, so split the sopranos and altos and tenors and basses into one and two. Okay, that’s alright.

Christopher S. Penn – 09:55
Okay, now score each candidate according to our rubric.

Ruby King – 10:05
“I did great. Thanks for asking.” Oh, it’s quite mean.

Christopher S. Penn – 10:23
So the first candidate scored 79 out of 100. The second one scored 82 out of 100. Great. Make a refinement of candidate two that improves the overall score. Create the candidate refinement and score it. So what we’re doing here — this is a three-part process that we use for generative AI when we’re doing anything commercially with generation, which is create, score, refine.

One of the things people do the most wrong with generative AI is to say, “Just do the thing,” and they accept the output — “Build!” — and they’re, “Oh, it kind of sucks. Generative is not very creative.” Well, no, you failed to prime the model. You failed to understand the context, and then you gave it a prompt that was stupid and insufficient. And so what we’re doing here is we’re basically telling the model, “You need to keep improving until you get it right.”

Ruby King – 11:33
That’s what my math teacher used to say: “Rubbish in, rubbish out,” too.

Christopher S. Penn – 11:37
So here’s candidate two refinement. Hungry eyes on Charing Cross. You can see it’s revised for better sound, revised for a smooth line, and so on and so forth. And so this refinement, it now scores an 88.

Ruby King – 11:50
Okay. Make it even better.

Christopher S. Penn – 11:54
Yes, essentially. See? Great. Take the refinement and improve it until it — improve it so that it scores better. So this is candidate set to refinement two, revised for alliteration and imagery, revised for more specific imagery, stronger association, stronger metaphorical language.

Ruby King – 12:24
Oh, it’s scoring a lot better now.

Christopher S. Penn – 12:26
Mm-hmm. And it tells you the justification.

Ruby King – 12:30
Hold on. I got eight out of seven there. That’s cheating.

Christopher S. Penn – 12:34
That is cheating. So it scored a 90. Now, the next thing we want to do — so every — almost every model that’s made out there, like Suno’s, has at some point, probably used a generative language model as its foundation. So what is — what are the characteristics? How would you tell someone else to compose a piece of chamber music?

Ruby King – 13:05
That’s very difficult. You mean lyric specifically?

Christopher S. Penn – 13:09
No, no, the music, the music itself. What would be the musicality? I’m going to save this — the lyrics themselves. I’m going to — great. Show me the final version of the refined lyrics without any of the notes for improvements. So what we want is just the clean lyrics. Okay, so we have our final lyrics here. That’s our song title, Hungry Eyes on Charing Cross. How would you give instructions to a composer to write the music that goes —

Ruby King – 13:58
— with this word painting? Okay, bearing in mind, what have we got? Beneath the gaslight amber gleam — the lines look about equal. So probably write a melody that aligns with the word painting of the idea of the piece because if it’s going to be repeated, they can’t work for every single line necessarily. Use that in the way it’s written in. It might just be an AAA structure here. It’s hard to tell because it’s just doing different sections.

Christopher S. Penn – 14:49
Okay, let’s ask it. See what it — what do you know about composing the musical to accompany these lyrics? Emotive, atmospheric, quiet introduction. Tenors and basses, soprano, alto, tenor, all voices. Is there instrumentation in chamber choir music? Is it all human singing only?

Ruby King – 15:08
There can be, but chamber choirs usually don’t have it because there’s so many different parts, and all the voices kind of work together. It’s a bit more like a string quartet. You can have other instruments with a string quartet, but the string quartet is, of its own, a string quartet that is capable of doing pieces of its own.

Christopher S. Penn – 15:23
Got it. Okay. What instructions would you give a composer to generate the music for this? Focus on technical specifics like tempo and key. So we’re going to essentially ask the model to distill down the instructions that it would give to a composer because when we get to the Suno portion, we want to feed it very specific instructions that hopefully will evoke the correct tokens in its memory to pull together its knowledge. The challenge is Suno’s prompting structure is intentionally very short so that you can’t give it highly specific instructions to replicate copyrighted music. So it has to be distilled down. So great. Distill down these instructions to 200 characters of text. Omit verbs.

Ruby King – 16:33
London night is not an instruction.

Christopher S. Penn – 16:35
Yes.

Ruby King – 16:36
Minor key is — tempo shift. Good word painting. Yes. Dynamics? Pianissimo, mezzo forte, fortissimo. No. Okay.

Christopher S. Penn – 16:45
Eliminate references to the content, such as London Night, so that our 200 characters of instructions are devoted to the music composition.

Ruby King – 17:03
Also, why does it want an unresolved ending?

Christopher S. Penn – 17:06
I don’t know. It seems to think that’s important. So that’s the unresolved question. Consider cello or piano optional.

Ruby King – 17:15
No, no, we’re not doing that here. Nice try.

Christopher S. Penn – 17:18
Okay, so let’s copy this into seven structures. We’re going to remove that because we’ve decided that’s not going to be something we do.

Ruby King – 17:25
It’s actually written out pianissimo, fortissimo this time.

Christopher S. Penn – 17:27
Yes, because we gave it more room. So this is 255 characters. So we might have to —

Ruby King – 17:34
— as long as Suno will — you do — Suno understands?

Christopher S. Penn – 17:38
Yeah. So we’re down to 227. Very texture. Let’s get rid of the unresolved ending. Now we’re at 201. Minor key.

Ruby King – 17:48
Chamber choir isn’t instructing — chamber choir. Elsewhere?

Christopher S. Penn – 17:53
Yeah, it’s right down here. So we’re at 187. So now let’s go into create mode. We’re going to do custom. We have — oh, it’s — sorry. It’s 120 characters.

Ruby King – 18:09
Okay. To be fair, the dynamic range doesn’t need to say that because that is — as quiet as you can go to as loud as you can get, pretty much. So just full dynamic range.

Christopher S. Penn – 18:17
So what if we do word painting? So you said full — just full dynamic range.

Ruby King – 18:21
It is, maybe, yeah.

Christopher S. Penn – 18:22
Okay, so we’re at 107 characters. Let’s do homophonic, polyphonic. That’s 151 standard chamber. Actually, just probably just chamber choir, right?

Ruby King – 18:38
Yeah, we’ll see what it does if it needs —

Christopher S. Penn – 18:41
— okay. Anything else that we could remove there other than that comma? We’re seven characters over still.

Ruby King – 18:57
When it has the dashes, is that saying, “Do it this, then this, then this”?

Christopher S. Penn – 19:01
Not necessarily. That’s just how it spit it out.

Ruby King – 19:03
Okay, then take out andante.

Christopher S. Penn – 19:09
Take out andante. Okay, so now we’re at 119, which is perfect. So let’s go ahead and take our — that’s going to be our song title. We’re going to put that down here as candidate one. Our lyrics — we’re going to get rid of the bolding. We don’t need it. Cut that into here.

Ruby King – 19:33
I can tell you’ve done this a lot of times before.

Christopher S. Penn – 19:35
Yes. And then we’re going to take our instructions for style of music, 119. Just to be on the safe side, I’m going to move chamber choir to the top of the instructions.

Ruby King – 19:46
Okay. I don’t want to rope a piece that is, “This cat is hungry.”

Christopher S. Penn – 19:51
And we’re going to create variant one. Is there anything that you would change?

Ruby King – 20:00
What, at this stage?

Christopher S. Penn – 20:01
At this stage, in terms of the musical instructions.

Ruby King – 20:08
Close, dissonant harmonies. It depends when it uses it. I think that’s a difficult one because dissonant harmonies do work as long as they’re resolved. If they’re not resolved — if it just closes, dissonant harmonies, then you don’t want to listen to it. I mean, dissonance is an interesting thing to use, but it’s only satisfying if it resolves.

Christopher S. Penn – 20:28
Okay. I pulled out homophonic, just went with polyphonic. So I did a second variation with that. Let’s see what it’s come up with for our variations. Are you ready?

Ruby King – 20:37
Yeah. Alright, I’m ready.

Christopher S. Penn – 20:41
That concludes part four. Thanks for tuning in, and stay tuned for part five! See you on the next one! If you enjoyed this video, please hit the like button. Subscribe to my channel if you haven’t already, and if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.

Fireside Chat: Music and AI, Part 4: Teaching AI to Write Lyrics: A Deep Dive into Chamber Choir Music

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest