ABOUT ROHIT: Linkedin | ROTH Capital Partners |

ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube


KEY HIGHLIGHTS

🍰 Have advertisers gotten more comfortable with ATT since early 2022?
🎱 How is probabilistic measurement doing?
🏗 Larger companies seem to fare better.
🤩 Facebook is seeing improvement due to the API.
💼 The value in iteration and history of ad accounts.
 🦋 The biggest changes in Facebook in 2023.
🥅 All about conversion modeling.
⚡ The rise of TikTok.
🤖 Generative AI on TikTok?
🚫 What if TikTok gets banned?
🍎 Has Apple’s stance towards mobile advertising changed from SKAN 2.0, 3.0, 4.0?

FULL TRANSCRIPT BELOW

Rohit Kulkarni 

Hello everyone, I’m Rohit Kulkarni, an internet analyst at ROTH MKM. Today, we’ll discuss a timely topic that has been on everyone’s mind in mobile advertising: what Apple has done and how is everybody else in the ecosystem adjusting to it. 

Shamanth Rao, who has technical expertise in mobile app advertising and traffic optimization will share his insights on iOS changes, return on ad spend and advertiser strategies for various channels like Facebook, TikTok, Snap, Google, and more. 

Shamanth Rao 

Thank you for having me, Rohit. I’m excited to share what we’re seeing in the mobile advertising ecosystem. Rohit, I don’t know if you have a preference for jumping off points.

Rohit Kulkarni 

What I would love to hear from you is when it comes to the impact of Apple and the war in Ukraine and various different external factors that affected all the advertising companies that we follow, 2022 was not a good year. 

So when you specifically peel out the IDFA changes and how every quarter progresses, maybe just talk about the lessons learned during 2022 and how we are set up entering into 2023 when it comes to measuring advertising performance.

Shamanth Rao 

In 2022, people held onto the older measurement paradigm, which is not sustainable.

The digital slowdown is mainly due to ATT. I was reading this piece that pointed out the Q4 sort of revenues and financial results. A lot of the non-digital companies, Walmart, Nike, all of these folks, their earnings, their share prices, were mostly on track, and a lot of their finances grew year on year. 

I can’t remember the exact numbers, but a lot of digitally focused companies did Facebook, Snap, and all of these folks, and suddenly, a lot of app-focused companies like Applovin, Unity, Playtika, all of those had very big downturns. A lot of a slowdown is precipitated by ATT, much less so by macroeconomic headwinds, because a lot of the offline companies were not hit anywhere as hard.

Rohit Kulkarni 

So in terms of advertisers adjusting to the changes, it seems like Q1 of last year was the first big quarter where people realized that things weren’t working. From an investor’s perspective, things may have improved as the year went on. But from your point of view, talking to clients, how did measurement and ad performance trend? Did it improve as the year progressed? 

From your lens, talking to your clients, how did measurement and ad performance trend? Did it actually improve? Or are we still at like Q3 Q4, 21 standpoint?

Shamanth Rao 

Measurement is unlikely to improve because it is dictated by Apple’s protocol. Although SKAN 4.0 was introduced, it is not a significant improvement in measurement. The only thing that SKAN 4.0 does is, if you’re already at high volumes, in terms of installs, purchases, and conversion events, you get less obfuscation of events. That is very marginal, and the overall SKAN protocol has not changed and I don’t think it will ever go back to having the deterministic accuracy that we had three years or four years ago. 

There will have to be non-deterministic, fuzzy measurements. There’s some amount of movement towards embracing non-deterministic measurement by people. But, measurement continues to be a casualty.

Rohit Kulkarni 

When it comes to moving towards non-deterministic or probabilistic measurement, what are the various ways that people are trying to do it? Is this still a work in progress? 

Or do people feel that these new ways of doing probabilistic measurement is something that people are getting more comfortable with?

Shamanth Rao 

It is a slow process as it requires a different skill set that many teams lack. For instance, I was talking to somebody who told me about booking.com. booking.com spends billions, and they have a team that’s dedicated to measuring the incrementality of everything they do, as they have a very dedicated team for this. 

Very few other companies have that level of dedicated resourcing. I’ve worked with public companies, and a lot of the analytics teams would not really focus on this problem on deducing the incremental impact of ad spend, that was not a project they were all focused on in the past. 

So I think the big hurdle really is the skill set of the people that are marketers, and their extended teams. 

But it’s been slower than you would like, just because it’s also harder to justify. A CFO would say, look, was my money on Facebook well spent? And if yes, by how much? 

3 years ago, you’d have said, we spent X dollars, we got back X dollars. Now, if you’re going to say, hey, we think there’s an incremental revenue of so much, but I can’t say that with deterministic certainty. It’s not nearly as easy a conversation to have. So I think a big challenge is organizational here. I do see a gradual movement and shift toward embracing these probabilistic models.

Rohit Kulkarni 

Is there a feast or famine situation where larger companies are now less worse off than much smaller companies that are still trying to figure out how to best allocate their ad dollars?

Shamanth Rao 

Yeah, 100%. So I think a lot of larger companies are much better off for a number of reasons. One, SKAN 4.0 builds the balance in the favor of larger companies because the more event data you have, the less Apple is going to obfuscate signals. 

Apple’s justification is that if you have more event data, people can’t reverse engineer private information from that. So, if you’re getting 10,000 installs a day, it’s much harder to reverse engineer one person’s data, as compared to when you’d be getting 10 installs a day. So if you’re getting 10,000 users, we’re not going to block your conversion value. 

Number two is, if you’re in a larger company, you are able to cast a wider net in terms of the channels you use, and you can be untethered from SKAN. So, I was actually talking to somebody that was looking at Rovio’s financials, and they were like, “Yeah, we still continue to spend 250K per day on all our games”. For user acquisition, a lot of that does go to non-deterministic, non-SKAN channels, which could be, influencers, or partnerships. 

If you’re large, and if you’re catering to a mass market, you can afford to spend on channels like influencers, which are not directly trackable. In addition, larger companies have the resources to conduct non-deterministic measurements on all of these channels, which puts them in a better position overall.  

Rohit Kulkarni 

What Facebook has been communicating to investors with regards to certain model conversions and different tools that they have implemented over the last 6-12 months, are getting progressively better results for advertisers. 

So what is Facebook doing exactly from your standpoint? And when a certain advertiser does or doesn’t use Facebook’s tools, what’s the delta there?

Shamanth Rao 

Measurement continues to be broken but performance has improved. Facebook is seeing improvement as it is using conversion API on the e-com front to ingest more data, so they’re able to maximize the signal that they have. They also have had a lot of automated campaigns, which automatically optimize for different assets within each campaign. But it is still limited by the conversion data that is passed on. 

So for instance, there is a game which gets very few conversions within the first 24 to 48 hours, Facebook still has a hard time optimizing for that. A lot of games, what they’ve done is if their conversions are happening after 2-3-4 days, they put in place a synthetic conversion within the first 24 hours just so that Facebook gets more signal. 

Part of why Facebook’s improved is just because advertisers engineered their systems so that they could give more signals to Facebook. Facebook has also been enhancing its automation tools and pushing its conversion API to more advertisers, who have been adopting it. All of these factors have contributed to improved performance on Facebook’s platform. 

Rohit Kulkarni 

Is that because of the conversion APIs that they have implemented, or are there any other reasons that the performance on Facebook you think, has improved during 2022?

Shamanth Rao 

Conversion APIs are the single biggest part of it because performance is really a function of the conversion event volume that Facebook receives from an advertiser. If an advertiser just isn’t sending any purchase data, Facebook cannot learn. That was really the biggest damage that ATT did, which was to cut off that stream of purchase data, or seriously damage it. 

Conversion API really did resuscitate a lot of that conversion data stream, so Facebook has an increased amount of signal to optimize. But there are other contributors like Facebook shops where Facebook does not have to rely on conversion data coming in from Apple. So obviously, the conversion stream is intact. 

Rohit Kulkarni 

Are there situations where an advertiser that worked with Facebook and they have continued to experiment with all these new tools? And after the first round of advertising, they spend a week trying to learn from the second, third, and fourth rounds. By the time they are done with maybe a few months of testing, the advertisers that are going through these iterations are actually coming out, sharing more data and coming out much farther along, than people who are experimenting for the first time ever, first time customers with no data and no history on Facebook. 

Is there value in iteration and history? Is that also helping, for example, the performance that we could potentially see, in Q1, Q2? The same advertiser would be better off in Q2 than Q1, because of the history that they are building with this new system.

Shamanth Rao 

Yes. The account history is a big determinant of performance and that is something the Facebook reps are very open and vocal about as well. So if there’s an advertiser with almost no history of performance, it’s just much harder. But it’s become even harder post-ATT. For  example, if you’re a meditation app pre-ATT, you’re completely new. Facebook would show your ad to users of other meditation apps, and purchases of other meditation apps. You would be able to get performance ramped up fairly quickly. 

Post-ATT, Facebook can still show your ads to users of other meditation apps. But the probability with which they can identify users and high-value users of other meditation apps is not very high. So it’s much less accurate now. For a completely new advertiser, there is a ramp-up period that is much slower than it used to be pre-ATT. 

The other variable is, for somebody that’s been running and iterating on an account, it’s not just the algorithm that’s learning. But it’s also the advertiser that’s learning about the ad creatives that are working. So for some of the advertisers we work with, we’re spending multiple hundreds of thousands, we test anywhere from 15-20 ads per month. So we have a much better idea of what’s performing, than somebody who’s run no tests. 

Rohit Kulkarni 

Of the changes that have happened in 2023 so far, what have been the biggest coming from Facebook in the last 90 days or in the next few months? 

What do you see from their sales guys as the big potential changes that they might be doing?

Shamanth Rao 

They are looking to integrate generative AI into their campaign dashboards. But, having played around with other tools, I can envision how this could play out. 

It could be, you enter a prompt for the user you’re looking for and they’re going to generate images and ads based on the kinds of users you’re looking for. That can be a huge game-changer. Because even though Facebook’s tried to make things as frictionless as possible lately, just in the last couple of years, a big hurdle for a lot of advertisers is still ad creative. 

If you’re looking at a lot of small indie companies, they don’t have a designer on the team which is holding them back. 

I was talking about creative learnings. A lot of advertisers are running three ads for the last six months and have not changed anything. If Facebook is able to integrate a generative AI tool within its campaign dashboard, that can unlock a lot of learning and efficiencies at the top of the funnel and bottom of the funnel in terms of what ATT has done, there’s no way to get that performance back. But you can make up for that performance loss by making the top of the funnel much more efficient. I think generative AI is one of the bigger things that I’m hearing about from Facebook, in terms of what’s on the roadmap, as we go ahead.

Rohit Kulkarni 

I’d love to repeat these topics on Snap and TikTok as well. 

In terms of just conversion modeling, what is that? I’m assuming it is still a probabilistic way? Is this the best thing that Facebook has done since sliced bread?

Shamanth Rao 

It’s been surely helpful for advertisers to get modeled conversions because it gives them some amount of confidence in the numbers that they’re seeing. At least it gives them something that they can report to their teams.

I think the biggest value in model conversions is helping advertisers manage the optics of their marketing reporting. 

So what conversion modeling means is, Facebook looks at the signals available to it, which includes conversions given by SKAN, but also other metrics that are available, which could be click-through rates on the ad, install rates on the ad, what percentage of the video was watched, and also who watched the video. 

So if they know that this user Rohit installed a meditation app and that Rohit might have made a purchase in a different meditation app, I can assign a very high probability to Rohit purchasing a different meditation app. 

So it is a probabilistic model that looks at the identity of the user that is perfectly known to Facebook, the behavior of the user on the Facebook platform, which Facebook notes with deterministic accuracy. 

Facebook has a somewhat incomplete picture of the events coming from SKAN. Facebook looks at all of these and says, SKAN says this campaign generated 50 conversions. But because I know who Rohit is, and he made purchases in other meditation apps, I actually think Rohit might have made a purchase here, so I’m going to count Rohit as a conversion in this campaign as well. 

I do want to talk a little bit more about the optics, as that’s a big piece of why it is important. So, with Facebook, a lot of modeled conversions are at least directionally correct. They match SKAN, and if SKAN metrics go up, modeled conversions go up. 

The modeled conversions aren’t as much in line with the performance on Google, where the performance is dramatically over-reported. Google dramatically over-reports conversions. That’s because Google uses its own analytics, it doesn’t even rely on SKAN, it takes SKAN’s data, but it doesn’t report on SKAN. 

You’re going to model this based on Firebase and the result is their CPAs and conversions look outrageously good. Oftentimes, we run non-deterministic models, like media mix models and we think Google’s claiming a lot of credit that they shouldn’t. The advertisers have been spending a lot of money because they trust Google’s numbers. 

But these are not true conversions, it helped the advertisers manage their optics. Therefore, we’re going to spend a lot of money, which is also another reason why I think getting non-deterministic measurement is ever more critical now.

Rohit Kulkarni 

The mix of spend that sometimes advertisers need to optimize on, how has that trended? 

If the first dollar goes to Facebook, then what does the next penny go to? Or how has that pie been broken up over the last six months? 

Are you seeing any shifts, be it from Instagram to TikTok or be it TikTok coming back to Instagram or anything like that?

Shamanth Rao 

There’s definitely been a significant shift towards TikTok. Part of it is that a lot of advertisers that had gotten spooked went over to TikTok. I know a lot of folks we’ve worked with did try Snap. The optimization wasn’t nearly as sophisticated on TikTok. 

You do see the instant virality that you still don’t see on Snap. So TikTok has definitely done much much better. TikTok has been a big gainer in the last 6-12 months. The TikTok team has been fairly aggressive in pushing stuff.

For instance, TikTok has a program that says, you get free Tiktok creatives made by partners like us. My team is a part of this program. Any advertiser that spends 20k and more gets free creative from us. They have to commit to spending 20k obviously. 

Rohit Kulkarni 

In terms of where that money is coming from, do you have that line of sight into if that advertiser is willing to pay 20,000 to TikTok? Is this something that they would have otherwise spent on Google, YouTube, Facebook, Instagram, or Snap?

Shamanth Rao 

A lot of these folks are people who wouldn’t have considered TikTok and they were running on Facebook and on Google. Creatives were a hurdle for them with TikTok. They’re like, “Oh, we want to try TikTok, but we really can’t get dancing videos. Our team can make animations and designs, but we really can’t make animated videos.” That was a big hurdle. 

What this program enables them is just to say, we’ll take some of our Google budgets or Facebook budget and put it in TikTok. In many cases, it’s huge for very large advertisers, so really, the dollars came as test budgets on new channels. They were diverted from the existing channels.

Rohit Kulkarni 

Okay, and how is advertising performance measured on TikTok as anecdotally what we have heard is, it is still more of a reach and frequency play less of a bottom-of-funnel play. 

Is that what you are seeing? You tend to work with more performance guys, So how is TikTok handling the problems that Facebook already has had for the last 18 months?

Shamanth Rao 

TikTok has the same problems because TikTok plugs into SKAN. We are seeing significantly improved performance on TikTok. It is not just a reach and frequency thing, like you pointed out, everyone we work with, is performance focused. So performance has improved. And what I would attribute that to, is just creative. I think that’s the primary piece. Just to unpack that a bit more, with creators,

a lot of advertisers have become more comfortable with TikTok-style creative. 

A lot of advertisers have more learnings from TikTok-style creators. The TikTok team has been fairly vocal and aggressive in pushing tools, not just for advertisers, but also for creators. They have something called Spark Ads, where if you’re an advertiser you can get an influencer’s video, and plug that into your ads with consent, of course. 

They have a TikTok ads dashboard, which is somewhat similar to the Facebook ads library. We get a lot more human support from TikTok. With Meta, it’s just very hard to get humans to talk to us. With TikTok we get much more direct support, and much more direct troubleshooting. So I would just attribute it to all of those factors that the share of TikTok has grown. The incentivization certainly doesn’t hurt.

Rohit Kulkarni 

Okay, three questions on TikTok, and I would love to touch on Snap later.

One is, do you feel now maybe this is like speaking in terms of second derivatives but obviously TikTok took share away from Google and Facebook, maybe more from Facebook last year. The amount of share that they would take away from Google and Facebook for people who already experimented a lot on TikTok, is that going to be less incremental? 

If somebody spent $100,000 on TikTok last year, and they’re going to spend $200,000 on TikTok? Where’s that extra 100,000 coming from? Is that still coming from Google or Facebook? Or is that coming from some other places?

Shamanth Rao 

In most cases, I would see this coming from Google and Facebook.

I would also point out that a lot of performance advertisers don’t operate with fixed-sum budgets. If performance is strong, they will find more dollars. 

So, if Google, Facebook, and TikTok are doing well, they’ll find extra dollars to put into all three. It’s not like they will cut one to put dollars into another, assuming the performance is strong. Obviously, if the performance isn’t there, then one will have to direct budgets to the other.

Rohit Kulkarni 

Second is on generative AI on TikTok. Is there anything that they are doing that Facebook is not in terms of helping advertisers and getting the creatives up and running?

Shamanth Rao 

The teams tell me they’re working on this. It’s just inherently harder on TikTok. Because all of the videos feature humans, all of the ad creative features humans. Whereas on Facebook, that is not the case. It’s much easier to make a generative AI-based ad for a T-shirt than it is for a parent with a kid who is dancing.

Rohit Kulkarni 

Last is on TikTok getting banned, or any likelihood of that – What’s your take? Do you feel that the advertisers are adjusting in case TikTok gets banned, or they just don’t care about all the regulations?

Shamanth Rao 

What I’m generally seeing is, advertisers aren’t panicking yet. I think the sentiment is that TikTok is too big to ban. But I was just pointing it out to somebody else – India has banned TikTok. That’s close to a billion people. So TikTok is not actually too big to ban, although politics is above my pay grade. But, if there is political will, it can be done. I have seen it before and this is something I’ve actually talked to multiple advertisers about, but I don’t see people making contingency plans yet.

Rohit Kulkarni 

If you want to reach people in Montana, who are using TikTok, you won’t be able to do that. Because I guess Montana banned TikTok last week.

Shamanth Rao 

The question is also about enforcement. How are you going to enforce that? Especially at the state level, at the country level, you could do draconian things as India has indeed done. Even then people use VPN and other tools to get through in India.  An interesting side effect of that, in India seems to have been an increasing adoption of Instagram reels, an increasing adoption of some of the local social networks, but it’s just hard to envision how that could happen at a state level or at a local level without being very draconian.

Rohit Kulkarni 

Where are we with Snapchat? With regards to the adjustments that they’re doing to iOS, the latest iOS changes, SKAN 4.0 tools and whatnot. 

Do you think they are at par with Facebook? Or if not, how far behind are they?

Shamanth Rao 

In terms of SKAN itself, it’s a function of what Apple sends everybody. There is parity in terms of SKAN. But in terms of what SKAN has been able to build in terms of conversion or model conversions, I do think they have been a lot slower. 

In fact, for a lot of folks we work with, it’s not been a hit. It’s not something we’ve seen a consistent performance from. Snap said “ATT is not going to hurt us. We’re going to be fine.” until the magnitude of the damage really hit them. 

In terms of the pecking order it was Facebook, Google, Snap, TikTok, a couple of years ago, pre-ATT certainly. Now that TikTok has overtaken them, Snap is almost like an afterthought for a lot of big advertisers. If there is a scale on the bigger channels, they do look at Snap, but it’s only if the other scale is on all of the other channels.

Rohit Kulkarni 

And is Pinterest on your radar? Where does that stand in the pecking order? 

Shamanth Rao 

It’s not on our radar. That’s because they deprecated their app-install product a couple of years ago, which was puzzling because we were seeing positive results for a subscription app that a friend of mine was spending six figures a month on. We were starting to see early signs of success and very abruptly they deprecated their app-installed product, so you can’t run performance-based ads for apps on Pinterest, even if you want it to.

Rohit Kulkarni 

The ad spend that you are seeing from your advertiser base, what is the outlook for the next 90 days? Is this something that you would see a step up in spend from Q1 to Q2? 

Shamanth Rao 

Q2 is usually slower for most verticals. So I would expect a slight slowdown as the algorithms improve, advertisers continue to adapt, and measurement becomes more accurate. 

However, Q2 typically sees a slowdown. Q3 usually experiences a small uptake, and Q4 picks up significantly.

Rohit Kulkarni 

Is there a way to quantify how far away we are with regards to just measurement or ad performance measurement versus pre-ATT days on Facebook. And is this something that you think they can catch up to? 

Are we at 50%-60% of ad performance? What does the trendline look like over the next few months?

Shamanth Rao 

It’s unlikely to reach 100% because deterministic data is not available anymore, and the pipeline is irreparably broken. Although Facebook’s conversion API and conversion modeling can make a difference, they can’t achieve 100%. 

But I don’t think the pre-ATT thing where Facebook knew exactly what Rohit purchased across 10 different apps, and 20 different e-commerce sites, therefore Facebook knew exactly what app to serve. All of that determinism has been replaced by some amount of probability. 

Certainly, the understanding that Facebook has is that Rohit can buy something that can have a high probability. It’s never going to be 100% in the way that it used to be.

Rohit Kulkarni 

Where are we now, 50%? 

Shamanth Rao 

Yeah, it’s definitely gone up from 30-35% in early 2022, to anywhere from 55%-60% now.  While improvements can still be made, there will be diminishing returns because the IDFA was the fundamental building block. If that’s out, it’s just really hard to have that determinism in measurement.

Rohit Kulkarni 

Has Apple’s stance towards mobile advertising gotten less worse in a way that if you look at SKAN 2.0, 3.0, 4.0, there could be 5.0 at some point in the next few months? So if you look at the progression over the last two years, how would you characterize it? 

Shamanth Rao 

Yes, there have been improvements in SKAN 4.0, but these have been very marginal. So for instance, they’re like, Oh, we’re going to offer you three postbacks, instead of in SKAN 3.0, you would get one post back only. That was with a 24-hour window, most typically. Now you have one after 48 hours, one after 7 days, and one after 35 days. Theoretically, this seems beneficial because you can track a user’s activity after 7 days and 35 days.

(A) It’s not cohorted data and (B) It uses a coarse conversion value , which basically means it can only be high, medium, or low after 7 and 35 days and it doesn’t actually have conversions. This means that the only meaningful data is the first conversion value, which doesn’t change much. So, yes, Apple’s offering bells and whistles. They haven’t been groundbreaking thus far.. 

Rohit Kulkarni 

Is Apple building its own mobile app advertising platform internally with better products?

Shamanth Rao 

They have offered a lot of new placements like a search ad, the product ad. However, I have seen people I know who have just stopped testing those as performance isn’t there.

A lot of the growth that Facebook saw from 2016 to 2021 was driven by SMBs, by mom-n-pop e-commerce stores that would spend $200-$300-$400 a day on Facebook. 

With Apple, with the new ad placements, the performance is just so bad that it’s just out of reach of anybody with a small budget. The only people who can afford it are maybe Candy Crush. It’s something similar to a TV ad. So, the new ad placements from Apple have been very ineffective for performance advertisers and for small and medium advertisers.

Rohit Kulkarni 

One quick rapid-fire question: If you were to choose Facebook/ Instagram/ TikTok/ Google, what’s your most favorite platform for advertisers now?

Shamanth Rao 

Usually, we begin with Meta in most cases because of its significant scale. However, for products that are highly visual and responsive to UGCs, we start with TikTok. But in the majority of cases, we prioritize Facebook, followed by Google, and then TikTok. 

In an increasing number of cases, TikTok is a second priority compared to Google.

Rohit Kulkarni 

Awesome. I wanted to thank you for your time and insights in terms of how things are evolving.  Thanks, Shamanth. Bye and take care.

Shamanth Rao 

Thank you for having me. Bye

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!