You Ask, I Answer: Why do Recommendation Engines Fail?

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

You Ask, I Answer: Why do Recommendation Engines Fail?

Oz asks, “Why is some consumer AI so bad? Instagram senses that I like bright-colored clothes. Then it shows me ads for bright-colored clothes that are also cheap crap that I’d never buy. What is the perspective of the companies?
– It works great for most people.
– We just need to get this right for 5% of people and that covers the cost.
– We know it generally sucks but it’s better than nothing.”

A lot of it is based on recommendation engines which have two issues – first, superficial data, and two, they’re a generation or two behind what’s current because of the enormous computational costs. Something like Netflix is going to use something like an LSTM because while it may not be as accurate, it scales much better than a gigantic, many-layer neural network that wouldn’t be able to update in real-time after you watched something.

A third part has to do with compensation model and objective optimization. What is the objective these ad systems are tuned for?

You Ask, I Answer: Why do Recommendation Engines Fail?

Watch this video on YouTube.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

In today’s episode oz asks, Why is some consumer AI so bad? Instagram, for example, senses that I like bright colored clothes, then it shows me add some bright colored clothes, they’re also cheap crap that I’d never buy.

What is the perspective of these tech companies? Is it doesn’t that work great for most people, we just need to get this right.

For five people, it covers the cost, we know generally sucks, but it’s better than nothing.

So it was a good question.

The answer has a lot to do with how recommendation engine technology works, recommendation technologies, take in datasets and essentially try and find patterns in those datasets to to predict outcomes, right.

So if we, if you like these certain things are going to predict certain other things.

recommendation engines can use a lot of very different algorithms under the hood.

And one of the challenges we have and it’s a challenge in the industry overall, is that a lot of these companies don’t reveal what is in their algorithm, what algorithm they’re using? Are they using something as simple as like, a naive Bayesian classifier? Are they using something as complex as you know, a many, many layer deep neural network? Are they using, you know, k nearest neighbor clustering? We don’t know.

We don’t know what’s under the hood.

And so we don’t we can’t necessarily offer input as to why some things behave the way they do.

But there’s two general considerations.

Well, three general considerations as to why some of these algorithms Don’t spit out useful stuff.

The first by far the most likely is computational cost.

The more complex the algorithm, the more it costs to run it.

And the cost here is in compute computational capacity, how fast can you get the result? With a lot of ad systems for example, you were talking about millisecond response times.

Particularly when you’re doing stuff like header bidding, and things where there is a real time auction going on.

And ad systems have to match and generate results extremely quickly.

And as a result, they have to pick algorithms that are super, super fast, even if the accuracy is leaves a little something to be desired.

I mean, it’s better than nothing.

For those who remember the early days of digital marketing, you’d be browsing on the website and you’d have like, you know, a Medicare wheelchair program being displayed to someone who’s you know, 22 and healthy.

It’s like, no, that’s completely incorrectly targeted.

Speaking of which, there is always the potential for advertisers themselves simply being so bad at advertising that they they have blanket targeting.

And all the machines cannot override a user’s preferences of the the advertiser says, Hey, I want to advertise to every living person within the boundaries of this nation.

Okay, as long as you got the budget to support it, it’s going to do that But computational cost is a big thing.

Second thing is what data you have going in the data that goes into the system may not be robust enough to offer anything that has true predictive power.

Especially if and this is important, especially if companies are correctly implementing ethical, unbiased AI.

You may not for example in a lot of cases judge somebody and you know, tune your ads on a protected class or you shouldn’t be let’s put it that way.

And so if the advertising that comes out is incorrectly targeted because you back end you know, ethical checker said, Hey, you can’t use racist as a targeting criteria for ads.

Okay, so now you’re gonna get, you know, Sham why, even if that’s not something that you want, because there may be some balancing happening behind the scenes to ensure that the protected class is not being used.

A third part is objective optimization.

And this is where this is where advertisers should be a little bit concerned.

Objective optimization and compensation models dictate how advertising networks work.

What does the ad network get paid for? They get paid for the impression.

Do they get paid for the click? Do they get paid for the outcome? advertisers have been pushing to very little success over the last 20 years with digital marketing to have average to have a action based or outcome based advertising where you get paid for the lead generated you get paid for the form filled out, you get paid for the shopping cart filled.

And understandably, the big ad networks have absolutely zero interest in doing this because it means much more rigorous computation on the back end, it means much more in depth tracking.

There may be substantial risks to the ad network because yet You could potentially, inadvertently or intentionally be collecting sensitive protected information.

And frankly, most ad networks realize that behind the scenes, ad performance across the board is pretty crappy.

I mean, we think about it.

When you look at like the click through rates on some of these ads, you know, look at these campaigns, you know, when people celebrate like crazy when they get like a 5%, click through rate, which when you think about means you wasted 95% of your budget, right? If you didn’t get more than 5% of the clicks.

From the advertiser perspective, you’re like, well, what did I pay for? If these systems were tuned to results only? advertising? It’d be a very different and much worse calculus for the ad networks because they wouldn’t get paid unless they got the result.

Is there a possibility that companies could pivot that way, potentially.

But right now, everything in advertising is effectively cost per impression when you look at the back end reporting and you see All these metrics in like Facebook stuff, effective cost per click, now what you’re really doing is you’re, you’re still doing all your bidding by impressions.

And you’re still doing all your optimization on that.

And as a result, it doesn’t really matter to the ad network, whether or not you click on the thing beyond with a reasonable doubt, but for the most part, it doesn’t matter because they’re getting paid on the impression, not getting paid a click for the most part, then definitely getting paid on the action that was taken.

Now if advertisers forced ad networks to to pivot and said, Look, we’re not going to pay you unless you deliver results that would drastically change.

The machine learning outcomes that allow these systems are tuned on, it would make them computationally much more expensive, because you would have to be, you wouldn’t be able to do simple stuff like k nearest neighbor clustering, just on on impressions, right? You would have to collect a lot more data, you’d have to collect a ton more data.

And that would make for a very, very different optimization.

When you look at how, for example, LinkedIn works versus how Facebook works for this advertising, LinkedIn stuff works very differently because they have as one of their major outcomes, we need to keep people on this professional network so that our enterprise talent management software, which is 40% of their revenue, can draw useful data from people’s profiles and recommend it to recruiters.

It’s a, that’s an example of a system that is much more outcome based.

And as a result, you see a very different culture on LinkedIn, you see very different advertising on LinkedIn.

Whereas Facebook is like, show all the show every ad possible, see what people click on.

Cool, great, whatever.

Same with Instagram, we get paid on the view.

So who cares what the result is.

So that’s why consumer AI is so sometimes untuned there’s a bunch of different reasons and there’s no way to clearly tell without independent third party audits, what’s going on behind the scenes, how it’s working.

I would love for company He’s like Facebook, for example, to reveal, hey, this is how we do the thing.

These are the inputs.

This is how the system is optimized.

But that is literally their secret sauce.

It’s unlikely that they would ever reveal that even if they could.

Because in a lot of cases, some of these in Facebook’s case, their neural networks are so complex.

I doubt there’s any single human could that could even interpret what’s going on behind the scenes.

The models are just that big.

So really good question.

There’s a lot to unpack in here about how these algorithms work, how they’re tuned, and what’s going on underneath the hood.

Hopefully as time goes on, we will see advertising itself pivot more towards results based advertising as well.

If you have follow up questions, leave them in the comments box below.

Subscribe to the YouTube channel and the newsletter.

I’ll talk to you soon take care.

One helps solving your company’s data analytics and digital marketing problems.

This is Trust insights.ai today and let us know how we can help you

Machine-Generated Transcript

Comments

Leave a Reply Cancel reply

Pin It on Pinterest