Note: We have a growing community of mobile marketers!

 The Mobile Growth Lab Slack: A community that was a part of our workshop series – The Mobile Growth Lab, is now open to the general public. Join over 200 mobile marketers to discuss challenges and share your expertise. More details are available here: https://mobileuseracquisitionshow.com/slack/

If you’re ready to join the growing community, fill this form: https://forms.gle/cRCYM4gT1tdXgg6u5

PANELISTS: Sherry Lin | Nick Blake | Gadi Eliashiv | John Koetsier | Shamanth Rao

ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

🎬 Preparing for Google’s Privacy Sandbox

🤾 How does the Privacy Sandbox differ from the SKAD network?

🧩 The biggest challenge in preparing and adapting to the Privacy Sandbox

🏵 Is fingerprinting here to stay?

🚧 Why has conversion model management been difficult?

🏖 Have targeting or creative strategies changed post ATT?

🌅 What are data clean rooms?

🎁 How has incrementality worked for the industry?

🧮 Are marketers leveraging Media Mix Modeling?

🔍 Role of an MMP in future

🔈 Retargeting solutions that can be helpful

KEY QUOTES

What does Google’s Privacy Sandbox look like?

Nick: We are going to get privacy, which the industry as a whole agrees is the right direction, but advertisers are not going to have to blow everything up and start again. They are getting ways that they can measure and we’ll get deeper insights than we currently have with SKAN.

How Privacy Sandbox differs from SKAN

Gadi: What is the resolution that you can break down your data in SKAN? You’re limited to 100 values on your campaign ID, that’s roughly seven bits of information. Google is enabling you to use up to 64 bits, which is insane. It’s like 18 quintillion options, if you think about it now.

Marketers can get more data with the Privacy Sandbox

Gadi: Obviously doesn’t really mean you’ll get all of it because they want to use differential privacy. But you’re definitely not going to get 100, it’s going to be a lot more than 100. What it really means is that marketers want to usually have metrics or dimensions, like campaign and creative and keyword and have this information available. So they can slice and dice the data, I’m pretty confident you will be able to do that with the Google solution. Whereas with SKAN, it’s very difficult today with 100 values, it’s hard to put that in.

The challenge in preparing and adapting for Privacy Sandbox

Shamanth: I am fairly certain that Google is going to privilege its own inventory, its own network. And they are already doing that. Even though Google doesn’t own SKAN, they don’t care about SKAN, if you want to attribute on iOS, you’re going to have to use Firebase. And they will give extra credit to their own conversions. With a Google owned platform like Android, that’s going to be way more magnified. That’s the one thing I’m very concerned about.

Why conversion model management has been difficult

Nick: One of the limiters of that is the 24 hour timer. Now, if you want to start optimizing towards something that’s a bit further down the funnel, you have to use one or more of those to keep that timer ticking over. So you need some upper funnel events to come through to make sure that you can get the owner to give you something to optimize the further down the funnel. So on the whole, we’re seeing a blend across both of those, we’re seeing a mix of people using those 6 bits, some for the upper funnel events. And some for those down funnel events.

Have targeting or creative strategies changed post ATT?

Sherry: On the retargeting side, it’s been really rough. Half of our audience can’t be retargeted because per ATT, we can’t be sharing any of these identifiers, not even server side ones like email addresses and phone numbers unless we get user consent. So our retargeting program has taken a really big hit.

What is a data clean room?

Nick: they’re safe places as they like to be called, where a lot of the bigger walled gardens are the sounds like Google and Facebook, they will be able to share with an advertiser that they both share customer level data, they bring them together, whilst they are still able to exert some control over that data as well. And the advertisers will be able to match the data that they have with the data from the platform. And they can see whether the datasets match up if there are any inconsistencies. Are you over serving one audience, under serving another. 

So as a brand, you would get much needed access to the data bit in a compliance space, because they do have features that make sure that the data that leaves can’t be tied back to individuals, so none of the data leaves that clean room, or the only data that leaves the clean room is aggregated, there’s no PII infringement or anything like that.

How incrementality has worked for marketers

Sherry: Incrementality is super useful, super important, especially for advertisers, again, that have high mindshare. So if you’re already getting a lot of organic demand, it’s especially important for you to separate the results of your ad spend, versus the people who would have come anyway. Did I order Starbucks because I saw an ad yesterday, or would I have ordered it today anyway, without seeing the ad. So incrementality has always been super, super important for us.

It’s important to prioritize with incrementality

Nick: It goes like this, in product management you’ve got good, cheap and quick, you can have two of those, but you can’t have all three. So you have to decide which two you want. And that is the thing with incrementality.

Media mix modeling using a basic spreadsheet

Shamanth: We’ve certainly done different types of media mix modeling, just including a basic spreadsheet based model, it’s not going to be any better or as sophisticated as what Coke or Nike would do. But we get an R squared which I think is great for two hours of work on a spreadsheet. In fact, we actually put a YouTube video about exactly how we did it. We also used Facebook’s Robyn, which is considerably more accurate. It was, again, surprisingly easy for us, compared to what we expected at least. Again, this was not perfect, it certainly requires a critical mass of data to be effective. I think it’s just a huge improvement over just using SKAN. So I would just encourage people to try both. Robyn is easier than you think and for the spreadsheet, you don’t need anything specialized.

A hybrid version of media mix modeling could work better

Gadi: Some companies are claiming they could do real time results to an extremely granular level. And it just doesn’t fit with how media mix modeling works. Somebody mentioned Nike as an example that uses very advanced, complicated media mix modeling. It’s also very service heavy, right? So I think there needs to be a hybrid, somewhat SAAS or Excel or something that is easy to do. I don’t know if you can expect it to give you real time data and maybe other folks have cracked it.

FULL TRANSCRIPT BELOW

John Koetsier

Welcome to the Future of UA Measurement webinar. So much has changed in user acquisition in the past 18 months. Everyone is wondering, when is this going to end? Where’s it going? What’s it all going to look like? And will it all suck as much as it has lately? We’re going to talk about iOS, SKAN and Android’s Privacy Sandbox. We will talk about some next generation measurement solutions, all of which are updated, modern buzzword compliant AI enhanced data science solutions. 

First, I want to introduce our talented panel, starting with Sherry Lin, who is a Group Manager at Lyft. We’ve also got Nick Blake, who’s the VP of EMEA at Liftoff. We have Shamanth Rao, who is the Founder and CEO of RocketshipHQ, and we have Gadi Eliashiv, who is the CEO and co-founder at Singular.  

Gadi, before we dive into all the content, who is Singular? Why are you putting this webinar on?

Gadi Eliashiv

We are a growth company and we help some of the world’s leading companies to manage their marketing data and solve for attribution and analytics. If you’ve been following our content, you’ve seen that we speak a lot about privacy. We’re very passionate about this. It’s our job to help marketers figure out how to handle these changes. Excited to dive into today’s topics. 

John Koetsier

We have a number of polls coming up, because we want to know what you’re thinking. The first poll is about our first topic, which is the privacy sandbox on Google. Have you started to prepare for Google Android privacy sandbox? Yes or no?  Are you fledging? SDK sandboxing? Topics APIing? Or are you just checking it out? 

Here are the results. 53% are just reading about it. 34% are not even thinking about it right now. It’s so far in the future. 13% are starting to get into it. 

Sherry, let’s start with you. As you know, Google announced its Privacy Sandbox. It’s actually less than two years from now, because they announced 3-4 months ago that they’re taking a different approach than we’ve seen with Apple and ATT. What’s your reaction?

Sherry Lin

When we first heard the announcement, I can’t say we were surprised. We knew that it was happening. I felt like we were still dealing with the ramifications of ATT. But the second thought was that it’s not too bad. Google was giving us plenty of heads up. And it seems like they’re prepared to be pretty transparent and take industry feedback. Their solutions address both attribution and targeting, whereas SKAN only addresses attribution. So we were cautiously optimistic.

John Koetsier 

Cautiously optimistic. Love it. Nick, your thoughts? 

Nick Blake 

From our point of view, we’ve got more runway, which is great. There is more structure, and there’s more communication than we’ve had with ATT. It’s been a bit more thought through from an advertiser’s perspective. 

We are going to get privacy, which the industry as a whole agrees is the right direction, but advertisers are not going to have to blow everything up and start again. They are getting ways that they can measure and we’ll get deeper insights than we currently have with SKAN. 

It’s a step in the right direction, and it’s an improvement  from our perspective.

John Koetsier

Shamanth, I want to bring you in here because I love your perspective. You’re helping dozens of apps and publishers all the time. What are you seeing here? What are you hearing?

Shamanth Rao 

Definitely unsurprised by the announcement  because we did expect Google to follow Apple. What’s reassuring is that it’s still in a somewhat distant future – 2024. It’s still somewhat limiting compared to what we have today. It’s good to know exactly what is going to unfold, and like Sherry pointed out, retargeting is included, and attribution is so much stronger. Gadi and John wrote about this on a blog, which I thought was very clearly explained. I definitely think it’s miles ahead of what Apple has done.

John Koetsier  

Shamanth I love that you brought up retargeting because I’ve talked to a bunch of marketers who have said so many people don’t do retargeting because it can be very complicated, very complex. And here’s a built-in system sort of in the OS level way of doing it. That’s a great point. Gadi, your initial thoughts, your reactions?

Gadi Eliashiv  

My first reaction was, wow, this doesn’t suck. It’s actually not that bad. It’s not going to suck for marketers. It is actually going to be good and not as disruptive or limiting. There’s a two year plus timeline, which is a lot of time. Seeing how third party cookies are still not canceled, they might take more than two years. We have a timeline we can work with. And the last point is, Google is actually asking for feedback from everybody. It’s almost a complete contrast to the SKAN launch. Google had a second mover advantage because they saw what worked and didn’t work for Apple but there’s so much drama with SKAN network release. There’s been a lot of panic over the last couple of years. 1.0 looked horrible and then 2.0 came about, slightly better. This is a different approach to releasing which I think everybody appreciates more. Hopefully, Apple will learn from this as well and iterate.

John Koetsier 

 How does the Privacy Sandbox differ from the SKADnetwork?

Gadi Eliashiv  

There are a lot of differences. A quick background for folks who don’t fully track that: it’s a change Google did, similar to what Apple did in iOS. So it’s coming from Google. It’s not a European legislation, or GDPR, or anything like that. I’m going to focus on the attribution aspects in the reporting. Similarly to the SKAdnetwork, there’s a concept of a real time postback. The device decides on some attribution, and there’s a real time postback. In that realtime postback however, the Google solution has more, in terms of granularity, for example.

What is the resolution that you can break down your data in SKAN? You’re limited to 100 values on your campaign ID, that’s roughly seven bits of information. Google is enabling you to use up to 64 bits, which is insane. It’s like 18 quintillion options, if you think about it now. 

Obviously doesn’t really mean you’ll get all of it because they want to use differential privacy. But you’re definitely not going to get 100, it’s going to be a lot more than 100. What it really means is that marketers want to usually have metrics or dimensions, like campaign and creative and keyword and have this information available. So they can slice and dice the data, I’m pretty confident you will be able to do that with the Google solution. Whereas with SKAN, it’s very difficult today with 100 values, it’s hard to put that in.

The other aspect is how can you report on user activity? Google is a bit more restrictive but they enabled something really massive. You could send multiple postbacks and what the user did over time, and this enables people to track what happens to a user, say seven days later, or up to 30 days later. 

That’s massive, because today, with iOS, you only get one shot. And you also have this timer system you’re working against. So it’s very unpredictable. You have to guess what the user is valued at within 24 to 48 hours. And it’s really difficult. You have to use predictions. That’s something that Google said they are going to solve. They are going to give you multiple postbacks, which is great.  That’s some of the differences. 

The last difference is that Google also has an entirely different mechanism called aggregatable reports. It basically means that you can get privacy friendly reports that have more data than the real time post backs. But the idea is that you aggregate them. Therefore, that’s why you can get more data because Google knows you will be aggregated. You can’t necessarily get it in real time. You can’t attach it to a specific user. Therefore, it can give you more data. You’ll even have a new reporting interface, not an interface like API or some way to get that where you can accumulate more data. So you wait a bit longer, but you get better insights. They’ve taken two approaches. I think they upped the bar a bit. Like Sherry said, it can work and it’s privacy friendly. It doesn’t suck. They’re trying to get there, which is impressive.

John Koetsier

Shamanth, what do you see as the biggest challenge? It’s out there. It’s in the future, but it is coming and losing the advertising identifier is a big deal. What’s the biggest challenge in preparing and adapting to the Privacy Sandbox?

Shamanth Rao 

I am fairly certain that Google is going to privilege its own inventory, its own network. And they are already doing that. Even though Google doesn’t own SKAN, they don’t care about SKAN, if you want to attribute on iOS, you’re going to have to use Firebase. And they will give extra credit to their own conversions. With a Google owned platform like Android, that’s going to be way more magnified. That’s the one thing I’m very concerned about.

John Koetsier

That’s interesting. And I have been poring through the privacy sandbox trying to find out, if you’ve put out something you probably want to make it good for the rest of what you do. Haven’t totally found it yet. But I think there’s some aspects there. Shamanth, we’ll dig into that in a future webinar.

Sherry Lin 

For Lyft as an advertiser, one of the many challenges is knowing when to jump in. A lot of these solutions require engineering resources for implementation, and data science resources for evaluation. So when is the right time? When is a product mature enough for us to go in there? If it’s too early, it’s not ready or there isn’t the right time to test, and also making sure that we line up the right engineering and data science resources to do the validation justice.

John Koetsier  

Thank you, Sherry. Let’s bring up poll number two now, because we’re going to transition into talking about the current state of SKAN API network. Poll number two is how would you rate your success with SKAN? You’ve been using it for 12 to 18 months. Has it been good? Is it passable? Do you really have some work to do? How would you rate your success with SKAN network from Apple using ATT? Has it been good? That would be great. Is it possible that it’s not amazing,  but you’re getting something out? Or is it really shocking right now and you need some work to do to get it?  

41% good. 50% I really have work to do. That is interesting. Thank you for those. Okay, so we’re gonna jump into SKAN as we know, but we’re gonna hit a hot button first. And everybody knows about it. iOS 16 is coming. Shamanth we’re going to start with you. There’s been a lot of fingerprinting in some segments of the market. Is that viable? Is that not viable? And do you think it’s going to go away? 

Shamanth Rao  

It’s definitely viable right now. What it does is it gets advertisers confidence that they’re making their money back. Whereas with SKAN, even if they’re getting their money back, there isn’t that level of confidence. But I doubt if this is going to be sustainable, longer term. You know, Apple explicitly calls out fingerprinting, and it’s very likely Apple will introduce a tech based enforcement. Something like Google’s SDK runtime, something based on private relay. But don’t think this is going to be around for too long.

John Koetsier 

Interesting that you say that and hopefully, that will be paired with some advancements to SKAN network itself, so that you have the confidence in the data that you’ve got. Gadi your thoughts there?

Gadi Eliashiv 

One thing that helps this kind of level setting is why fingerprinting could even be used.  Based on our data, a ton of spend goes to publishers like Facebook, TikTok, Snapchat and Google like the self attributing networks. And that’s an area where even if the advertiser or anyone else wanted to fingerprint, they can’t because of the way it works. It limits the amount of fingerprint inventory in a way to something much smaller. It’s helpful because it means you can think about it. The SKAdnetwork is already the majority of the spend. And it’s been out there for a year plus, right. So it’s not like some nascent technology no one’s using. 

Of course, everybody’s suffering but it’s already running out in the wild quite extensively. I think that the fingerprint is definitely not viable in the long run. I agree with Shamanth, definitely viable now, not in the long run, all indications are saying that Apple will work to stop it very soon. And then again, to be fair, I’ve been hearing indications for the last two years. No one here is an Apple Oracle. They’ll do it when they want to do it. But I do think this solution will be more technical in nature, we kind of have some evidence for it, because you could see that their last solution was private relay and iCloud Plus. And there’s some thoughts like, Hey, maybe they’ll adopt this SDK sandbox Google made up, maybe they’ll do something similar. My bet is some technical solution..

John Koetsier

My bet is somewhere around private relay. Because the SDK sandbox is pretty new. I think Apple will love a solution like that. But I think it’ll take more time to build that into all their infrastructure, and everything. 

Let’s turn towards conversion model management, because a huge percentage of you on the webinar are saying it’s not good. Conversion model management has been one of the toughest challenges. Nick, let’s start with you.

Nick Blake 

With SKAN, it’s quite difficult. People are still using the six bits approach which was better than the 64 different options.

One of the limiters of that is the 24 hour timer. Now, if you want to start optimizing towards something that’s a bit further down the funnel, you have to use one or more of those to keep that timer ticking over. So you need some upper funnel events to come through to make sure that you can get the owner to give you something to optimize the further down the funnel. So on the whole, we’re seeing a blend across both of those, we’re seeing a mix of people using those 6 bits, some for the upper funnel events. And some for those down funnel events.

Something that we have been developing which will come out soon as a predictive LTV modeling option, where we’re starting to take away some of the proxy and level events that we would do two hours a day with seven dollars. You’ve got your day seven over here and your day 365 or something over here. We’ll be able to optimize further up with real events. But at the moment, it’s still very much that it’s a bit limiting. And it’s that mix, and people are trying to understand which of the many events that they have are the ones that best correlate. It is a lot of trial and error in using the six bit approach.

John Koetsier 

Thank you for that Nick. Shamanth, where have you seen it work? What’s working for conversion model management?

Shamanth Rao

We’ve tried to keep it simple, at least for the small to mid size developers we’ve worked with. As long as we are able to maximize signal in the first 24 hours, ideally, a purchase signal for subscription after the trial signal. It almost always is good enough, not perfect, it’s not deterministic, it’s good enough to make SKAN work. For the apps that have not enough signals in the first 24 hours, that can definitely be a challenge.

John Koetsier 

And it’s got to be a particular challenge. Just SKAN and the entire framework for the small to medium size publishers that you’re talking about because of the privacy thresholds and the missing information. Smaller campaigns are more adversely impacted. 

Sherry, that seems to bode well for a big player like Lyft, what’s worked for you in terms of conversion model management?

Sherry Lin 

We don’t have as much of the data scarcity issue. But I would say, in the poll, we would be in the we’ve- got-work-to-do bucket. We definitely still have lots of work to do. We haven’t been testing the conversion models as much as we should. Right now we’re sending all of our funnel events; Apple, signups, searching for a ride, activation, which is defined as a first ride. We are sending all of these events to SKAN. And that allows the marketers to test which of these signals are actually good for optimization. Ultimately, we’re trying to optimize for cost per activation, cost per first ride. But right now we’re still seeing performance not being at par with Android not being at par with where it was before SKAN. So we definitely still have work to do.

John Koetsier

Interesting and sometimes you gotta go on faith based marketing. Hey, our organics are way up. Why could that be interesting? Something good must be happening. Sherry, let’s stick with you. Have you changed targeting or creative strategies as a result of ATT and SKAN?

Sherry Lin  

Not so much on the UA side, we’re still optimizing towards cost per activation. But

on the retargeting side, it’s been really rough. Half of our audience can’t be retargeted because per ATT, we can’t be sharing any of these identifiers, not even server side ones like email addresses and phone numbers unless we get user consent. So our retargeting program has taken a really big hit.

But we are looking at some privacy centric retargeting solutions out there. Distillery is something that we’ve been looking at. There’s the new seller defined audiences. There’s topics from the privacy sandbox. The latter two are really nascent, but we’re definitely keeping a close eye so that we can jump in and experiment whenever they’re ready.

John Koetsier  

Will SKAdnetwork 3.2 come out with something like FLEDGE. So you can have retargeting built into iOS. Could that happen? Okay, here’s the interesting part. We’re going to get into the future of measurement in just a moment, incrementality, media mix modeling, data cleanrooms and all that stuff. First, we’re going to have our chance to chat with Tim Cook at WWDC which is coming up. You’ve got five minutes with Tim Cook, he cares about what you’re going to say, he’s going to listen to what you’re going to say. What is the one feature request you have for improving SKAN for marketers? Sherry you are up first.

Sherry Lin 

The retargeting one actually was a good one. But I would also say for attribution, the ability to do incrementality testing is sorely missing in the world of privacy in the world of SKAN, where incrementality is so important for bigger advertisers, especially those who get a lot of organic traffic to begin with. And we haven’t really figured out a way to do incrementality testing in this new world of a scan of ATT.

John Koetsier

Good one. Shamanth, your turn is up. You got your five minute slot with Tim Cook. He cares. He’s listening. What do you tell him?

Shamanth Rao

Get rid of the privacy thresholds.

John Koetsier 

Then he comes back and he says, “Well, what do you want? Because we’ve got to make it private still.” What do you say to that?

Shamanth Rao

I think the timer allows for privacy. 

John Koetsier

Okay. Interesting. Nick, you’ve got your five minutes with Tim Cook. What do you say?

Nick Blake 

I’d probably first speak on behalf of a lot of our clients that say, Can you keep fingerprinting? We know he’s gonna move away from that. I’d asked him for more communication, more consideration around the advertisers so that it’s not this guessing game that everyone is having to make about what are we going to use? What aren’t we going to use? So my big one would be communication.

John Koetsier 

Tim Cook comes back and says do you know Apple? Did you think we were Google? 

Gadi, do you have the time you have been waiting for? You have your five minutes with Tim Cook? What are you saying?

Gadi Eliashiv 

Knowing how harsh Apple will be I’m probably really going to lower my expectations. So I’ll try and ask for something. If they can extend SKAN to support Web to app, I think that will be useful. And it should be super easy and not going to compromise the privacy, just need to add some parameters to the URL. You send people from the website to the App Store. It’s just adding some parameters, because there’s gonna be a really big gap. Once and if they enforce fingerprinting, even on their own media, where some people interpret it as, it’s my website, stay out of my business. And they’re like, no, it’s not your website. It’s our iOS platform, we can throw everything right. That’s the debate. So once they really tried all these measures, you’re going to lose some of these very basic capabilities that might not even have to do with advertising. It’s just like people on your website and your app, you want to know it’s the same person. So maybe SKAN kind of helps. It would be nice if  SKAN could be expanded for that.

John Koetsier  

Excellent. There is a new functionality that Singular has released, SKAN advanced analytics. Can you introduce that? Then we’ll bring up poll number three.

Gadi Eliashiv 

One thing that we’ve noticed is how hard SKAN can be. And even the basics, just getting testing done is difficult. But once you get all this stuff in place, you realize that the basic framework that you get is not enough, you need to figure out what data Apple’s hiding from you with the privacy thresholds, you need to figure out the right model for the conversion and iterate and you can’t build cohorts. And so we had to work really hard. 

We just released this new capability that is trying to address a lot of these issues, we model conversion values, we automatically look at your conversion models and suggest new ones that might be better and more optimized. And we also use prediction to reconstruct cohorts. It’s the first time I’ve seen us work really hard to rebuild capabilities we used to have in the past. But that’s the new reality of privacy. We’re rolling out to our customers. There’s a lot of use cases where this can be incredibly powerful. So if you’re interested and you’re struggling with conversion models, and you want to look at cohorts, please talk to us, we’d love to consult and help.

John Koetsier 

Here’s poll number three. Do you want to learn more about SKAN advanced analytics? How does it help optimize your iOS measurement? There’s a yes and there’s a no. 

We’re going to switch gears to the future of measurement. We’re going to talk about data clean rooms, incrementality, media mix modeling and all that stuff that we think is going to be a big part of the future here. So we’ve been hearing about data cleanrooms. 

Results for poll number three – we got 85% saying yes. Lots of people want to know more. 

Nick, we’re hearing about all these new technologies, one of them that’s been around for some time, is a data cleanroom. What is that? And what is the value there?

Nick Blake

As you said, they’ve been around for a while and in different guises or different names. I remember Facebook, running them with clients many, many years ago. I think it’s probably something that’s newer to us in the mobile app world. But ultimately,

they’re safe places as they like to be called, where a lot of the bigger walled gardens are the sounds like Google and Facebook, they will be able to share with an advertiser that they both share customer level data, they bring them together, whilst they are still able to exert some control over that data as well. And the advertisers will be able to match the data that they have with the data from the platform. And they can see whether the datasets match up if there are any inconsistencies. Are you over serving one audience, under serving another. 

So as a brand, you would get much needed access to the data bit in a compliance space, because they do have features that make sure that the data that leaves can’t be tied back to individuals, so none of the data leaves that clean room, or the only data that leaves the clean room is aggregated, there’s no PII infringement or anything like that.

Now, the platforms aren’t doing this out of the goodness of their heart, it obviously works for them, because they don’t have to part with valuable targeting segments that could be then used by that advertiser on a competitive platform. So they are able to grab a bit of share of wallet as well from their rivals. The barrier to this is, you need a lot of data, they’re not cheap. The two data sets will be in different formats. So there’s a lot of prep work, a lot of hours that are needed that go into this. 

John Koetsier

Gadi any thoughts on data clean rooms?

Gadi Eliashiv

I think Nick’s answer was wonderful. And maybe I’ll add just a couple of things, There’s different flavors of cleanrooms. And one really cool thing that Google is doing with their Privacy Sandbox is, they created this component called an aggregation service, which is basically this special program Google will provide and you will run on your own servers, like the MMP will run it for you. And it will receive encrypted data with a lot of information. And that service will decrypt an aggregate, so it’s sort of a cleanroom, in a way, because you get a lot of touch point data that you’re not supposed to have otherwise. 

And because you aggregate it on the fly, and you can’t trace back to user level, you get insights. So it’s a twist on it. We’ve also been working on our own version of the data cleanroom, for a couple of years now, we actually have something called Private Cloud, which is basically like a private instance of our software. And we found that if you go through that, it’s quite the setup, because you want to separate all the components in our stack to a private server, then you can have more of a dialogue with publishers about hey, this data is truly segregated from everybody else. It’s my own customers, private servers, etc. 

But I have to say, at the end of the day, a lot of it comes to publisher participation. If you don’t, we can’t get extra data from Facebook, Google, etc, then, this stuff relies on all of them wanting to do that. I will say that I’m seeing some encouraging movement with some of the big publishers, and it’s also this game where they say, can you ask your customers to ask us, and then we get a lot of demand, and we might be more open to it. It’s in that step of playing that game right now. And, without sharing stuff, there’s some people making a bit more progress. But it’s still early, I agree with Nick, though. It’s been around the web forever. In mobile, it’s still a bit more nascent.

John Koetsier 

I think Google’s aggregation servers that it talks about in the Privacy Sandbox are kind of a similar concept in a way as a separate service. There’s some aggregate data that came out of there. Obviously, it’s different in terms of per brand and all that. But there are some interesting things there. I want to turn to incrementality and Sherry you’re going to be at first with this one. That’s fitting because you talked about that as a need that you felt you had. So, what do you do? Where are you on incrementality? Is it workable? Is it workable for performance? Is it workable for mobile marketers? Is it useful?

Sherry Lin 

Incrementality is super useful, super important, especially for advertisers, again, that have high mindshare. So if you’re already getting a lot of organic demand, it’s especially important for you to separate the results of your ad spend, versus the people who would have come anyway. Did I order Starbucks because I saw an ad yesterday, or would I have ordered it today anyway, without seeing the ad. So incrementality has always been super, super important for us. 

On its own, though, it’s not helpful for performance marketers. Because the data comes in slowly, you have to do the study, you have to do the analysis. And then you come out with a coefficient that you then apply to your attribution. So on its own, it’s too slow, and, and it’s too slow for the performance marketers to make their decisions. It’s also really expensive. It takes a while to run to get to stats, especially if you already have really high organic demand. Getting that stats to signal to show up is especially difficult. We still try to do it everywhere. But it always has to be done together with attribution to work for performance marketers.

John Koetsier  

Excellent. So Shamanth, your thoughts on incrementality? For performance marketing, mobile marketing?

Shamanth Rao 

It’s definitely something we’ve embraced in different forms and flavors. By that, I mean, oftentimes, it can just be monitoring blended pre-posts, doing basic spreadsheet based models, or using third party tools to measure incrementality. It’s definitely been much, much more valuable, just because SKAN is just worthless.

John Koetsier  

Tell us what you really think, Shamanth.

Shamanth Rao

I think in one of our previous webinars, we talked about how some CPAs are 500 or 1000. And you just can’t compare it with another which has a CPA of 50 or 100. You need to have incrementality in some form, because of SKAN attributions as well. But I think what I would just caution about is just that the incrementality analysis is going to be only as good as the underlying model. We’ve tried tools that on paper, spit out an answer. But the models are not accurate enough so they are just meaningless. So I would rather have something that’s somewhat accurate than something that just isn’t.

John Koetsier

That’s such a risk. You just get a few things wrong as you’re doing your incrementality measurement. And you know, you’re a month in and boom, it’s not gonna work or the world changed or whatever. 

Sherry Lin

Incrementality is difficult, especially like you said, when the world changes, like during COVID times, the incrementality, results from six months ago, will be really different, especially for ride share. It’s so difficult. 

Nick Blake 

There’s different arguments around the methodology as well. It goes like this,

in product management you’ve got good, cheap and quick, you can have two of those, but you can’t have all three. So you have to decide which two you want. And that is the thing with incrementality.

John Koetsier 

Wonderful, it’s a great time to bring up poll number four and pull number four, guess what it is about – incrementality. Are you leveraging incrementality right now? Yes, I’m actively using it. No, I’m not or three we’ve tested but not on an ongoing basis. 

So let’s have those results come up for incrementality. Are you leveraging incrementality? Yes, I’m actually using – 13%. No – 55%. Thank you so much for sharing. 

Okay, let’s talk about media mix modeling and Sherry you’re going to be at first with that one just because you’re awesome. Media mix modeling is not new. We know that right? It’s been around for some time. I think it’s over 100 years old, the actual concept and early implementations of it. It’s gathering some attention now. It’s a potential solution, will you be leveraging it?

Sherry Lin

I think this is an area where we do poorly. We have tried media mix modeling in the past, but because our business is so geo focused, depending on the supply and demand in each city, and the government regulation in each of the cities, or these contractors or not, it really impacts our media spend. So it’s been really difficult for us to have a good media mix model that works because of the instability. And also our spend is very localized, and also very stop and go. So we have not been able to make an MMM work for us. But I’m very curious to hear from other advertisers that they’ve been able to target with them, especially app based advertisers.

John Koetsier 

I think she’s handing the baton to you, Shamanth.

Shamanth Rao 

We’ve certainly done different types of media mix modeling, just including a basic spreadsheet based model, it’s not going to be any better or as sophisticated as what Coke or Nike would do. But we get an R squared which I think is great for two hours of work on a spreadsheet. In fact, we actually put a YouTube video about exactly how we did it. We also used Facebook’s Robyn, which is considerably more accurate. It was, again, surprisingly easy for us, compared to what we expected at least. Again, this was not perfect, it certainly requires a critical mass of data to be effective. I think it’s just a huge improvement over just using SKAN. So I would just encourage people to try both. Robyn is easier than you think and for the spreadsheet, you don’t need anything specialized.

I think we might actually have YouTube videos on how to use Robyn as well. You can go check those out.

Sherry Lin  

What do you think about during COVID times when things are so unstable? 

Shamanth Rao 

No, I don’t think that would be applicable. Because again, I think there has to be some precedent. So no.

John Koetsier  

Let’s turn to Gadi on that just briefly, and maybe Shamanth you can do some quick Googling, find your video and drop the link in chat there. 

Gadi Eliashiv 

Incrementality has been getting popular because of all the privacy changes. So it makes a lot of sense. Like folks have said SKAN is a lot more limiting. So you’re looking for additional signals. And I think overall that’s how it’s going to be used as additional signals, it’s not going to rip your signal from SKAN or privacy sandbox, but it’s something else that you use to inform decisions. And it’s something I foresee also becoming more prevalent in platforms like Singular. I think you’ll have something that will eventually grow across the board. It’s clear that Facebook has interest in media mix modeling, because of the work on Robyn, which is an open source framework. 

I think once someone like Facebook is interested, they’re going to push it and judging from the past, whenever Facebook cared about a particular way of measuring things that usually became more distributed and more available everywhere else, I still have to be somewhat sure how quickly information can be usable.

Some companies are claiming they could do real time results to an extremely granular level. And it just doesn’t fit with how media mix modeling works. Somebody mentioned Nike as an example that uses very advanced, complicated media mix modeling. It’s also very service heavy, right? So I think there needs to be a hybrid, somewhat SAAS or Excel or something that is easy to do. I don’t know if you can expect it to give you real time data and maybe other folks have cracked it. 

John Koetsier 

This is a good segue. Take a couple minutes before we go into our top takeaways from every panelist and then the question and answer segment. What is the role of an MMP in the future? Right now a few years from now two years from now. There are so many different sources of insight, the challenge of getting all this data from different sources and creating a clear signal out of it keeps increasing. What’s the role of an MMP in a couple of years? 

Gadi Eliashiv 

I would then say the need, and the role almost hasn’t changed. Companies must grow. They use that data to guide marketing. You need measurement and analytics, you need to make right decisions. And in that aspect, MMPs role remains the same. Our job is to provide data measurement insights, just simplify what is otherwise a very complicated landscape. Everybody knows it got a lot more complicated in the last couple years just means our job might be harder. And if you look at all the privacy changes, you get SKAN with a million limitations we just talked about, you’ve got a lot of cross device flows, some even talked about that in the chat, you could send people to a website and then your app, and you can do all sorts of things. Android’s going to have a massive change, third party cookies are going to be deprecated. So definitely a lot of changes that are coming. 

On top of that, there’s other critical data points like your marketing spend, and what people do inside the app, and you get deep linking. And all of that is, it’s just information we have to process and make useful. We also talked about additional methods like media mix modeling, incrementality. So I think it’s clear that the level of complexity is increasing. I had this stupid analogy, and I didn’t know if I should use it like, it’s like the spring suspension. So the road is really bumpy. You kind of want to make the road ahead for the marketers a bit more stable, right. And so our job is to make that simple, to absorb all the changes and make it a bit more simple. But yeah, the technology and complexity is increasing. The complication is increased dramatically, which is exciting for the engineers, and less exciting for development costs and everything. Our job is the same, we have to provide insight. So marketers could keep doing their jobs. There’s a lot of opportunity there, there are a lot of signals that we didn’t care about before because IDFA was around and it was easy, but a lot we’re going to use an app. 

John Koetsier 

We’re gonna go into our high level takeaways. If you remember one thing from this webinar, here’s what it is.

Sherry Lin

Block and tackle the immediate measurement challenges to keep the lights on. Your business has to keep going so do what you can to keep your programs running. However, also look to the future and think about what ultimately is the right thing for your users. If they expect often that you need to honor that, it’s really difficult to try and write that line and break your promise. So always think about what is ultimately right for your users.

Nick Blake

Embrace the change and don’t get too comfortable. As we’ve all been discussing here, things shift a lot. If things are working well don’t get too complacent. If they’re not, don’t get too downbeat. There’s opportunities and things are going to move around a lot. 

Shamanth Rao 

Embrace fuzziness. I think we talked about it quite a bit. You’re not going to have perfect measurement. Get comfortable with that.

John Koetsier

I love that word Shamanth. I’ve started to use a little bit in one of my recent blog posts – fuzziness. You may not know what the truth is, but if you can get a high degree of fuzziness, then you can proceed confidently on that and optimize and grow and all that other stuff.

Gadi Eliashiv

Some folks missed the SKAN train and woke up with what station am I at? I love conversion models, so this is like a second chance? You get a two year head start, you can get ready. I’m sure it’s gonna change a lot. Work with advisors and your internal team, just don’t be complacent because it will happen. And clearly, everybody’s serious about privacy. 

John Koetsier 

Excellent. There’s a fifth poll just before we get into the Q&A. 

Why would I want to get better at SKAN? 93% say SKAN is not the solved problem. 

We have some questions from you in the audience. Sherry, I’m gonna send this one to you because the person is asking you, can you name the retargeting solutions you recommend? You were talking about how hard it was? And you said there were a couple solutions, what were those?

Sherry Lin 

So one is Dstillery. It’s a cookieless retargeting solution, where they use algorithms, basically machine learning to extrapolate from a seed list. So you still need a seed list of identifiers. And using your seed list, they can use algorithms to predict who else it’s basically a look alike. That’s supposed to be accurate. They’re claiming that it’s almost as accurate as one to one retargeting using an identifier, but we haven’t tested with them yet. 

Another one is seller defined audiences. This is a solution that’s being developed by IAB. And it’s kind of like the Topics API where each seller can identify their own audience categories, that allows the advertisers to bid on. And then the third is Topics API, which is part of the Privacy Sandbox, where Google defines these different groups of people that are alike, and then advertisers can choose to bid on them. 

John Koetsier

Wonderful. Thank you so much. We have a great question on Privacy Sandbox and I’m going to send that over to Nick first, because I’ve been super curious about this as well. With Google’s Privacy Sandbox, where will the ad auction take place? And we know that it actually takes place completely on device. And that makes me wonder a little bit what happens and how ad networks function with that. 

Nick Blake

In all honesty, I don’t have an answer for you.

John Koetsier  

Gadi, any thoughts on that one?

With Google’s privacy sandbox. Where will the ad auction take place? 

Gadi Eliashiv 

That’s a good question. They don’t, at least while speaking from the measurement standpoint, they don’t intervene as early as an option, but in some of their retargeting solutions, the idea is that you kind of tell the device, hey, I want to serve ads. And then you could use the functionality or the device will decide if they want to show a retargeting ad, or irregular ad. And I think it’s similar concept to the targeting itself. I do think that there is some sort of intervention in choosing the ad, but it’s a great question in how it will be implemented with having multiple bids and some of the decisions that device needs to make and some  sort of speed needs to make. So I’m not sure actually.

John Koetsier 

Yeah, it is a good question. I think it is open actually. And I’ve looked at it a little bit in a blog post I did for Singular on Topics API. I’ll find that link in a moment and share that. 

Shamanth – what additional signals of truth are publishers adding or adopting outside of last touch attribution. What additional signals of truth, what data signals are publishers and advertisers adopting?

Shamanth Rao

We’ve talked about some of those, just in terms of non-deterministic measurement and content out there. And certainly those things you’re looking at I think the other signals we’ll look at are very much more top of the funnel, which I think could be interesting. Which means for videos, what is the three second view rate? Which is something we have been looking at and did not look at, prior to ATT. Because we see how much of the video definitely says that it’s a fairly good proxy for the performance of the creative.

John Koetsier 

Excellent. I’m looking at additional questions here.

Why a DSP doesn’t support an SKAdnetwork? Nick, any thoughts? 

Nick Blake 

I don’t know at this stage. 

John Koetsier 

SKAD network signal optimization. What works? You’ve always got to work quite a bit on that to get the right ad.

Nick Blake 

So we’re still getting some signal. There’s an element of our ML that is still working, there’s some parts that are coming in. It’s dependent on what people are optimizing towards, and how we can fit that in. So it means that is a bit of a mix, dependent on the client, I’m afraid, which is a bit vague, but that is the actual truth.

John Koetsier 

Well, we’re gonna have to call it to a close. It’s been an hour. I want to thank everybody for being part of this. It’s been an amazing webinar

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify, Google Podcasts or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!