You Ask, I Answer: Why Do Open Source Large Language Models Matter?

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

You Ask, I Answer: Why Do Open Source Large Language Models Matter?

In today’s episode, Amy asks why open source AI models like LLaMa 2 matter. I explain how they allow independence from big tech platforms, custom tuning, and control over data privacy. Open source lets you run models on your own terms rather than relying on third parties. Tune in for a breakdown of the benefits open source provides businesses adopting AI.

You Ask, I Answer: Why Do Open Source Large Language Models Matter?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

In today’s episode, Amy asks, Can you explain in layman’s terms why open source models like llama to matter so much? Can you break it down so we can understand what it’s what’s important about? Okay, so the llama to model which was released by meta, still want to say Facebook, but it’s better is an open source model in that they have given away the model itself and the model weights.

What that means is that anyone who agrees to the license can take this language model and run it on their own hardware or software.

And if you have some fewer than 700 million users per month, you don’t have to pay Facebook meta, you don’t pay meta a dime, it’s free to use.

Now, when you look at existing models, like the models that power chat GPT or the models that power Google Bard, we have access to the interface, right, we can log on to chat GPT and we can start typing, and we’ll get some results for free.

And if you want more better stuff, you pay the 20 bucks a month for the the pro account.

But you don’t get the model itself, right, you get an interface to it.

That means that you are perpetually using someone else’s model on their terms in their system in their services.

And at any point in time, they can turn off the switch, and you no longer have access to it.

As we’ve seen many, many times with chat GPT when their demand is very, very high, those sorry, we’re having performance issues.

So the service is not running right now.

You can’t do anything about it.

You’re like, Okay, well, I guess I’ll just get a sandwich because I’m okay.

That’s a model owned by someone else.

An open source model.

And it’s there’s some there’s some debate about whether you can truly call it open source because a meta has not released the training data used to make the model, they only released the fully baked product itself.

However, there’s a whole very arcane system of decompiling and retuning models we’re not going to get into effectively though.

functionally, it is very much like an open source product, you can take this thing, you can modify it, you can run it, you can do all sorts of stuff with it.

And you don’t have to pay meta for it until you reach that 700 million a month user mark.

Why is this a big deal? Because if you want to integrate generative AI, large language models into your product into your product or service.

If you use a big provider like open AI or Microsoft Azure or Google, you have to pay for it.

Right? Every time someone uses a chat bot in your in your app, you have to pay a small amount of money to open AI or to Microsoft or to Google.

And that adds up real fast, right? It’s it’s fine in testing.

Oh, it’s three tenths of a cent to run this query.

Okay, great.

What happens when it’s 100 million users a day? Asking those questions, what happens when it’s, you know, people asking 80 questions at a time, those bills add up real fast.

Also, at any point, you know, Microsoft or Google or open AI can say, Well, you know, I really don’t like how you’re using my software.

So I’ve cut off your account, which is really problematic when you think about it, because it means that access to this technology is heavily controlled by large technology companies.

When you have an open source model, you download this onto your laptop onto your servers onto your compute infrastructure and you run it with open source infrastructure like cobalt or llama dot cpp or any of the the many, many, many, many engines that can run a large language model, they’re all open source and free.

At that point, no one can control you anymore.

No one can say, Hey, you can’t use the model for that because it’s on the privacy of your own laptop and never there’s no data going across the internet.

If you want to have a conversation with llama instruct, which is the chat version of it entirely about like clowns covering themselves in ketchup, you can you can have that conversation.

That’s completely private is completely up to you how you interact with the model.

From a business perspective.

Again, as long as you’re under that 700 million user a month requirement, that means that instead of paying hundreds or 1000s or 10s of 1000s of dollars to open AI, because you’ve integrated their software into your product, you run this model instead, and you pay the cost of your computer infrastructure running it instead.

So if you’ve already got a room full of servers, it’s not a huge stretch to add more and run the model yourself, then it’s yours.

If chat GPT is down that day, you don’t care, because your servers and your infrastructure are running your version of the model.

And no one can take that away from you.

No one can censor you.

No one can tell you can’t do this or that.

I mean, they can tell you they can’t enforce it.

And so that’s a it’s it’s about independence.

It’s about freedom of choice.

It is about control over the AI experience.

You can also take these models and tune them you can you can modify them.

If you have if you download the llama two base model, you can retune it and say, Okay, I want you to be specifically expert in accounting, you’re going to know general ledgers, double entry bookkeeping, generally accounted, accepted accounting procedures, you’ll know, tax structure, you’ll know all this stuff.

And you’ll reweight the model, you will influence it, you’ll modify what it can and can’t do, it will get worse at some things, right? It will, as you the more you tune a model towards a specific task, the less can do other tasks.

So it will stop being able to tell jokes as well.

But boy, will it answer any question you have about any accounting scenario, or any marketing scenario or any HR scenario, you name it, you can retune these models.

And so these are the reasons why, at least from my point of view, these open source models are really, really important.

They allow you to, to build your own infrastructure.

They allow you to tune a model to very specific circumstances.

And they give you more control, give you more control.

If your company deals with personally identifying information, sensitive, protected information or protected health information, you absolutely should not be using a third party model for those you should, under no circumstance be handing that to open AI or Google or whatever.

When you download and run the llama model, for example, from meta, yes, Facebook made it.

But it’s not on their servers.

It’s on your laptop and no data is being transmitted to meta.

It’s all being run locally, which means that you have control over the data that you’re giving to a model, what it does with it and where that output goes.

And for ensuring privacy and data safety.

That is the only choice that you can use in large language models and still comply with all the laws that you’re required to.

So that’s why it’s a big deal.

It will, it’s getting to be a bigger and bigger deal every day.

And it’s something that people need to be aware of, and to have in their mix of their generative AI solutions.

So great question.

Thanks for asking.

If you like this video, go ahead and hit that subscribe button.

(upbeat music)

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest