ABOUT SIMON: LinkedIn | Dave

ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

👁️ Why you should keep an eye on your app’s initial organic traction

🎬 When you should start investing in your startup’s UA functions

📈 The highest-ROI activities you should focus on initially

🗝️ Creative – a key component of UA

👴🏻 Why many companies still run legacy systems

👀 The best time to start hunting for the best MMP

📊 You really need a sophisticated reporting system

☝ The one piece of infrastructure you need to scale up

⛐ Why acquiring to your current LTV might not be the best strategy

🤝 How a MarTech manager can help in your UA post-ATT

👨‍👦 The best proxy for the attribution of the past

👍 Using SKAN data to get the best possible estimates

😥 Building a long tail of channels might not be a good idea

🙋 One of the first roles you should hire for

💪 Essential qualities to look for when hiring talent for your UA team

🔮 Agencies vs full-timers vs contractors 

🏃 What’s better than hiring someone junior just to execute

✔ Overseeing the agency relationship – things to watch for

🎨 When hiring an agency for your creative production makes sense

➕ Incrementality in a nutshell

🔌 What an early stage company can do with incrementality that a mature company can’t

KEY QUOTES

What you may face when moving from a big corporate to a start-up

You might have worked in a big corporate, where the marketing team was north of 40 people, and you had a person for every single use case. You had MarTech people, data analysts, experienced media buyers. And then, you go to a start-up and all of a sudden, you see that reports are being run on Tableau and they’re a bit obsolete and they’re not really useful for your use case. They’re not really getting things to the place they need to be, and it’s really problematic.

How did you get your UA buyers to develop expertise?

I brought in some outside help and set up a mentorship program for my UA buyers. I brought someone in for one hour a week for our key channel, the one where we spent the most of our money, just like everyone else, which is probably channels like Facebook, Google and Apple Search. The outside help mentored and tutored my people for a certain number of hours a week, and that actually helped us make a lot of progress.

A reason why experienced leaders may be wary of joining start-ups

I’ve consulted with companies that are at a pretty critical stage in their growth, and they secured Round B or C financing, and they’re big companies with substantial market valuation. But they’re still running, if not legacy, then Tier 2 MMPs and Tier 2 reporting. And their reason was that they never really had the resources. They grew too fast. And now they’re facing the problem of not being able to grow their business, especially on the marketing side, because they’ve had second tier tools and second tier partnerships. This happens to the extent that someone of my caliber or even higher caliber than me, like the VP of growth or a CMO, who’s being interviewed for a role in such a company, might not even agree to join because they’re using obsolete tools.

Why you need a top tier MMP even in the early stages

I understand that if you’re looking at your revenue-to-cost equation on those tools, having 5% of your costs go towards your MMP is a lot of your overall costs in the early stages. But if you scale your marketing spend by 5X, all of a sudden that becomes only 1% of your costs, right? And without that MMP, you cannot get to that scale where a lot of it becomes efficient. So in a way, this efficiency buys you more efficiency!

Why acquiring to your future LTV makes sense

Let’s say you’re looking at the LTV curves of companies like Lyft and Uber. And you see that one year revenue per user early in the life cycle of the company is actually very little. But then the markets mature, and the service matures. Look at Uber. They added Uber Eats and other additional services. Now their LTV is 3X or 4X of what it was four years ago. But if they were acquiring to the LTV four years ago, they would have left a huge market share on the table today.

How your pre-ATT baseline can help you today

If you have a product with stable conversion rates that track over time, you always need to have a baseline to refer to. Let’s assume our baseline happened before ATT and we had steady conversion rates, and now our conversion rates have declined because we’re just not able to attribute users to certain channels such as Facebook, Google, Apple Search – basically, paid channels. So if you’re running paid attribution, you need to look at your baseline before all the changes and apply similar conversion rates to the post ATT environment. It’s a fairly simple concept, but it does work.

What to look for in your UA media buyers

That’s what I think of first for UA media buyers. Is this person good enough for my current level of spend on a particular channel, or can they take us to a different level if they come in with strategic experience?

Why work with contractors?

You don’t really need to have the approval of the entire team, for example. As opposed to hiring a full-time person, contractors are really like a nice band-aid. But even with contractors, I think long-term. If I’m hiring a contractor, can I potentially convert them into full-time? If I do convert them, will they stay with me for half a year? I don’t want to hire a person for a month – it’s just really not the best investment.

An advantage of working with agencies

The reason is that agencies, especially the good ones, employ highly experienced media buyers. It’s really difficult, especially in this environment, to secure that number of experienced media buyers. Hiring 5 people with 8 years of experience will take you forever in this market. Those people are in demand!

When it comes to incrementality, start-ups have an edge over publicly traded companies

If you’re a publicly traded company and you want to measure incrementality by turning off all your paid acquisition, then there is a cost to be paid there, in cost and efficiencies. When you turn things back on, your campaigns lose all the history, right? So they have to reoptimize again, and you can lose some money there. You’ll need to ramp up because you don’t know how platforms will behave. But if you’re in the early stages and you can afford that strategy, add a new channel, turn it on, turn it off. See what happens with your overall mix.

FULL TRANSCRIPT BELOW

Shamanth Rao

I’m very excited to welcome Simon Vishnevskiy to the Mobile User Acquisition Show. Simon, welcome to the show.

Simon Vishnevskiy

Thank you so much for having me.

Shamanth

We’ve known each other for many years now, and you’ve done some big things since we last connected, which we will talk about. What struck me as particularly inspiring and impressive is that you went into your current role and built a mobile marketing function from scratch without having much of a blueprint. And your current company has just had its IPO very recently, so the proof is in the pudding, so to speak. I’m excited to dive into all of that in your experience, Simon.

Simon

No, thank you so much, glad to share. This should be fun and interesting.

Shamanth

You joined your current role at Dave, and you set up an entire mobile marketing function. Now let’s say somebody is in a similar position while starting in a new company. They see there isn’t much infrastructure around in the company. How do they decide if they should set up a mobile marketing function, or whether the product is just so broken that they should just quit and go home?

Simon

As a lot of people who are listening to this will understand, you have to prove your product first. You have to get some organic traction. You have to understand the key metrics and you really need to know whether or not the product is viable, and whether you can get any kind of monetization at all on your products from your first organic cohort.

So setting up certain tests and KPIs, and figuring out whether or not you can drive towards those KPIs is the number one question. For instance, can you really get an MVP going and understand where the product is viable?

But then, you may have certain indicators and you may see your user base move in a certain direction. Your early cohorts of users are your early adopters. They’re very receptive people, and if you’re able to monetize them, then the next logical question is – can you actually start increasing your user base and start investing in some of your UA functions? That’s the point at which you have to make the decision. My first thought would be whether we are seeing enough organic traction, enough organic growth. Are there really good metrics from our users, such as retention, monetization, even early on in the product life cycle? That’s extremely important.

Shamanth

True. Beyond just product-market fit, there have to be some quantifiable metrics, like you said, retention, monetization, to understand if the product can actually benefit from marketing.

Moving on, tell us about when you started in your current role at Dave. Can you give us the lay of the land and tell us about what the infrastructure and the team was like when you started?

Simon

Definitely. One of the things I inherited was very junior UA buyers. And this is really a reality of the start-up world. You often have a lot of marketing spend, although you don’t necessarily have the resources to compete with the big companies. So you end up having very junior media buyers that are oftentimes in charge of massive budgets. So when you come in in a situation like that as the head of UA, you must definitely assess and understand whether you need additional hiring or additional training. What must you do to make your media buyers more efficient? For instance, you may choose to bring in an agency, and that’s something we’re probably going to talk about later on. So that’s one thing for sure – assess your personnel.

Apart from those junior media buyers, I also inherited a massive reporting structure, which is also something a lot of people get when they start somewhere.

That was also something I inherited.

Next thing is the MMP. Having the right MMP is probably one of the most important things along the lines of reporting. And that’s something I’ve inherited that I was happy about. We were working with AppsFlyer; that was the MMP of choice and we’re still working with them.

These are things you really need to consider when you’re starting – what have you inherited? Do you have good attribution? Because without it nothing works. Do you have good reporting? Because without it nothing works! How strong is your team? I inherited a mix of good and bad – the infrastructure was in okay shape, but my team needed a lot of training.

Shamanth

So you had junior media buyers, some not great dashboards, but you had a strong MMP. I’d also say that some people who start off at start-ups inherit no infrastructure, which comes with a different set of problems altogether! But certainly, I think hitting the reset button on infrastructure that isn’t working is a huge challenge.

Aside from what you inherited, as you got started in your first couple of weeks and months, what were some of the things you focused on? What were some of the things you didn’t focus on? What were the highest ROI activities in the very beginning and what were the lowest?

Simon

You know, that’s definitely a case of 80-20. I really like the 80-20 concept, and I apply it to a lot of different situations. When you’re starting, there’s almost always two things with respect to the highest ROI.

First is where the money is. So how are we optimizing? Where are we spending the money? Where are the gaps? For instance, is a certain activity channel-specific? Basically, day-to-day operations of figuring out where the biggest bang for the buck is.

The second is infrastructure and support. What is that other bucket that will help you spend your marketing budgets more efficiently slash scale? So you’re looking at the current situation, but you’re also considering infrastructure that can push you towards the future.

For us, there were a couple of things. The first on the infrastructure side was reporting, definitely. Without good reporting, you cannot understand how your money is working for you, where you’re getting your results—by channel, by portfolio and so on. What are you looking at in terms of the performance? You cannot really optimize and spend efficiently without it.

But then on the portfolio level as well, you really need to understand what the channel performance is like. How are you tracking it? What are some of the day-to-day optimizations that you can do?

So those are a couple of my key pillars.

The last pillar I haven’t touched upon yet—although it’s really the key to any good user acquisition program—is creative. How is your creative optimization? What’s your creative reporting like? I had to build a lot of infrastructure for all of these. On the creative optimization front, I had to pull a person from my team to build reports, and I had to work with one of our external partners that actually had a creative optimization tool and implement it in-house.

On the reporting side, I was able to secure some resources, albeit not as ideal as I hoped, but it was with our limited scope and capacity in mind. So I started fixing the infrastructure.

And then, as far as the day-to-day stuff goes, I did something different, which I think would be interesting for some of our audience because it’s not a typical solution.

I brought in some outside help and set up a mentorship program for my UA buyers. I brought someone in for one hour a week for our key channel, the one where we spent the most of our money, just like everyone else, which is probably channels like Facebook, Google and Apple Search. The outside help mentored and tutored my people for a certain number of hours a week, and that actually helped us make a lot of progress.

So those were the key pillars that I focused on immediately.

Shamanth

You’re right, in that the data infrastructure piece is so critical and yet it oftentimes gets overlooked, just because it also happens to be resource intensive to set up dashboards. It happens especially at early-stage companies, but you really do need to understand where your money’s going, and it’s easy to overlook that.

Which brings me to my next question. With building dashboards and ensuring data integrity, you could say there’s a case for simply using MMP dashboards and getting Excel reports from it, and then using that for day-to-day decision making. That is one possibility. So how do you decide whether a company should just use MMP plus Excel reports, or build a customized dashboard, be it in Tableau, an internal database, or third-party visualization tools? And does that actually change with the stage or scale of the company?

Simon

It’s an interesting question. Right. I don’t think there’s really a one-size-fits-all solution, but I can speak from my point of view. I’ve quite a bit of experience doing this in start-up companies and companies of a higher scale, so I’d like to think I have an interesting perspective on that.

My perspective is this – you need to do your best right from the get-go.

I’ve consulted with companies that are at a pretty critical stage in their growth, and they secured Round B or C financing, and they’re big companies with substantial market valuation. But they’re still running, if not legacy, then Tier 2 MMPs and Tier 2 reporting. And their reason was that they never really had the resources. They grew too fast. And now they’re facing the problem of not being able to grow their business, especially on the marketing side, because they’ve had second tier tools and second tier partnerships. This happens to the extent that someone of my caliber or even higher caliber than me, like the VP of growth or a CMO, who’s being interviewed for a role in such a company, might not even agree to join because they’re using obsolete tools.

They might refuse because this situation is really hard to come in and straighten out in a period of time.

So my answer is to use the best MMP right from the get-go. As you know, they’re tiered. Adjust and AppsFlyer are probably the two dominant players in the market now. And then below that you have things like Tune or Branch, right? So you should look at getting to that A-level right from the get-go if you have the resources, because that will drive your business – data integrity and understanding your true performance.

Next comes reporting. And this is the answer to your particular question of why we can’t just use an MMP and Excel. You should build a more sophisticated reporting system, because that feeds into a lot of different use cases.

It can be for building LTV projections, which is probably the north star for a lot of companies, right? It’s not just about understanding what you’re getting from your budget. You should know what you expect to get from your budget in a period of a year, two years or three years. It’s how we ran user acquisition in Zynga, as you remember.

So I would recommend having that robust reporting and having the right MMP right from the get-go.

With that being said, those top tier MMPs charge a substantial amount. Maybe you cannot afford it at your Round A or seed money level. You’ve got to start somewhere reasonable. But you should do it as early as possible, because otherwise, it really hinders your ability to scale.

I understand that if you’re looking at your revenue-to-cost equation on those tools, having 5% of your costs go towards your MMP is a lot of your overall costs in the early stages. But if you scale your marketing spend by 5X, all of a sudden that becomes only 1% of your costs, right? And without that MMP, you cannot get to that scale where a lot of it becomes efficient. So in a way, this efficiency buys you more efficiency!

So that’s my answer. Get tier one tools as soon as possible, and then build the best robust reporting as fast as you can and as early as you can in the company’s life cycle.

Shamanth

So what I’m hearing you say is that even if you’re a startup with a relatively low spend, oftentimes you shouldn’t just build for your current level of scale, but rather for your future level of scale, because you’re not going to scale otherwise.

Simon

That’s exactly right. And a similar concept applies to people acquiring to the current LTV too.

My previous job was at Lyft, and I’ll use that industry as an example.

Let’s say you’re looking at the LTV curves of companies like Lyft and Uber. And you see that one year revenue per user early in the life cycle of the company is actually very little. But then the markets mature, and the service matures. Look at Uber. They added Uber Eats and other additional services. Now their LTV is 3X or 4X of what it was four years ago. But if they were acquiring to the LTV four years ago, they would have left a huge market share on the table today.

So you should always have some assumptions about your growth. They could be extremely aggressive, which I don’t recommend, or they can be grounded in reality. And this is why you need to have the MMP and reporting in a similar fashion. You need to assume certain growth.

Shamanth

Yes, and I’ve certainly seen products with significant downstream LTVs but they’re more focused on acquiring D-60 or D-90 payback. They won’t be able to scale if they’re focused on D-60 or D-90 when all of their revenue is way beyond it. So I agree with your answer, it makes so much sense.

One of the things you said was that you built a lot of custom dashboards, including one to understand which creatives are driving performance. Obviously the world of data and analysis has changed massively since last April, since ATT; conversion value tracking is broken because of SKAN; privacy thresholds are obfuscating a lot of things; and there’s just no reliable creative level reporting. So how have you thought about modifying your dashboards in the light of ATT, given the completely broken ecosystem?

Simon

That’s an extremely relevant question, especially in 2022. We’ve seen ATT and we’re also seeing Google probably moving towards restricting the ability to track on the device as well. That’s coming in 2022 or 2023 – we can only guess the time frame, but it’s definitely coming based on the direction in which the wind is blowing, so to speak.

So ATT was definitely supposed to be a gravedigger, especially for iOS. There was a lot of confusion and definitely a lot of panic surrounding it. But if you are reasonable about the things you know you can do, you can definitely come up with a game plan.

There are a couple of things to my game plan. First—and this is in addition to infra and reporting—I added a MarTech manager. This is for our audience here, especially if they have the resources and are now at Round B or Round C, or even publicly traded companies (although the latter tend to have MarTech managers, especially if there are marketing budgets).

A lot of mid-stage companies don’t have a MarTech manager. And the value of this person is extremely important, because they will set up SKAN for you, which help you track in the absence of any other attribution and ATT. For us, we’ve pivoted out from not being able to track to running our user acquisition campaigns fairly efficiently, because we’ve been able to reproduce the SKAN networks and set up and work with AppsFlyer. This is really an important piece of tracking for us. We went from not getting any data to having enough data to be efficient on iOS and being able to acquire to iOS.

So that’s the first thing I recommend – get a MarTech manager, set up a SKAN network and work with your MMP, because this is extremely important. Setting up SKAN now is really the best proxy to how things were in the past.

The second thing is internal attribution. This is directly in answer to your question. How do you build dashboards that give you pretty much the same access to data as they used to before ATT? Of course, It’s not really a hundred percent possible to do it, but it can replicate 90% of that. One of the techniques and strategies is to do something called reattribution.

If you have a product with stable conversion rates that track over time, you always need to have a baseline to refer to. Let’s assume our baseline happened before ATT and we had steady conversion rates, and now our conversion rates have declined because we’re just not able to attribute users to certain channels such as Facebook, Google, Apple Search – basically, paid channels. So if you’re running paid attribution, you need to look at your baseline before all the changes and apply similar conversion rates to the post ATT environment. It’s a fairly simple concept, but it does work.

We’ve done this across all our iOS acquisition, and we’ve been able to reattribute users and establish a new baseline that is somewhat similar and accurate, as opposed to missing all those signals and not having them on our backend because of the restrictions in iOS.

It’s very important to do this internally because your MMP is missing all those signals and missing all that data. Unfortunately, your MMP cannot do much except to help you set up the SKAN network and attribute at least some of those signals.

So there’s multiple layers working here – first, the internal reattribution that you need to set up, for which you need to bring in your data science team. It’s not a sophisticated concept; you just have to establish a baseline. And then based on that historical baseline, you reattribute some of your activation, installs, whatever the key metrics are to your paid campaigns in the present moment. Set up  SKAN and do all the work on the SKAN side to be able to ingest some of the signals.

On the advertiser side, make sure that all the signals you have are being properly mapped. Then channels such as Facebook, Google, Snap, TikTok and whatever you work with are able to optimize off of the correct events that are still being tracked, because otherwise you’re going to lose a lot of performance.

So it’s really a multi-layer approach. I covered some of the key pillars and that should be helpful for people who are starting from scratch and don’t necessarily have an idea of how to combat this.

Shamanth

So if I understand you correctly, you’re saying if I know my Facebook conversion rate is 25% better than on Google, that’s 25% better than on Snap, and I have the install counts, then I can basically use my SKAN data to get an estimate of how many activations and transactions I got from Facebook, because you know, it’s 25% better than Google, and that’s 25% better than Snap. That’s very interesting, and I can see how it’s critical to have a MarTech manager who can do that analysis and use it to build dashboards to make it clear that, yes, even though this is an estimate, it’s a well-founded estimate.

Simon

That’s exactly correct. The best we can do is estimate, but at least this is an estimate based on your historical data. You have to apply that in the absence of better indicators and try to make sure that the data going forward is still being passed through SKAN, and you have some nuggets of data on which to base your decisions and optimizations.

Shamanth

I know you briefly touched on hiring and the team you inherited. You’ve obviously hired folks since then and your team has grown. What are some of the milestones at which you decided to hire new people? Is it linked to X amount of spend, or X number of active users?

Simon

If you’re heading operations for any company, there are really a couple of things you need to do. You must look at your current scale and you must look at your potential scale. What that really means is knowing how many more resources you need in order to get to a different level or squeeze out higher efficiency at your current level. These are really budget considerations – your goal, your current spend levels, whether your team actually has the expertise at the current level to do the work as well as whether it has the expertise to push us to a higher level. You need to do this assessment.

You must also understand opportunities for scaling and where those opportunities come from. Again, that’s another 80-20 question. You and I have worked together at Zynga, we ran the portfolio of games across different channels and you can attest to this. A lot of the time, companies focus on building the long tail of channels. For example, you bring in someone and they onboard seven or eight different channels. But in aggregate, a spend on those eight channels end up being 10% of what you’re doing on Facebook. So bringing in an experienced person on Facebook and optimizing that X amount of budget will give you higher yields than bringing in someone else and onboarding 20 different channels. So that’s something to keep in mind – where are your opportunities for growth? Where is the current spend and how does the person in that role support excellent optimizations and strategy—anything that really drives the company and the portfolio forward.

That’s what I think of first for UA media buyers. Is this person good enough for my current level of spend on a particular channel, or can they take us to a different level if they come in with strategic experience?

Then there’s infrastructure. Like the MarTech manager I mentioned earlier. That’s a key hire for me. I can tell you that if I did not hire my MarTech manager in my current role at Dave at the right time, our results, especially in the light of ATT, would have been completely different. We wouldn’t have been able to scale as much as we’ve done and feature in App Annie or Sensor Tower charts, because we wouldn’t be able to spend as we’d have no signals.

Similarly with reporting. If our reporting wasn’t as robust as it is right now, considering as it does things like ROAS, Day 7 return and some other downstream KPI funnel metrics, we wouldn’t be able to spend as efficiently as we are now.

So you need to hire for your infra, and on the day-to-day side, you want to make sure that your ambitions or your current state is really backed up by a person that has the necessary skills to either maintain or grow the business.

I hire with a growth mindset, not a maintain mindset. If I’m thinking of 3X scale, I want to know what a new hire understands in terms of strategy and execution, what they’ve done in the past, and the best methodology and best practices they know that will push us to that scale.

Shamanth

Speaking of hiring, what were the first roles you hired for and why?

Simon

So there are a couple of areas where, honestly speaking, a lot of companies do not invest in and are not even aware of it. As you know, 2021 has been a hot year for startups, and a lot of companies are well-funded. So a lot of people come in for growth leadership positions, and they simply don’t have enough experience. They’re not well-rounded, and they don’t understand the whole 80-20 conversation.

So for me, one of those areas was getting a person to do app store optimization. I think it’s important to do ASO the right way. When a company gains a bit of traction, look at how many people land on the front page in their app store, whether it’s App Store, Google Play. This is where the journey starts, where all user journeys start. If you have a mobile app, this is exactly where the user lands, whether they come in organically or they arrive after clicking your ads. That’s such an important part of the journey, and so not optimizing that part of the journey and doing it the right way is really a head-scratcher for me.

Shamanth

When you say ‘do it the right way’, what do you mean? I know there are a lot of ASO experts around, so according to you, what’s doing it the right way versus somebody who just doesn’t know?

Simon

Doing it the right way is having a strategy. And that strategy is testing, and having the tools to back up the testing. As a matter of fact, I’m working with a former colleague of ours, Sarah, whom we used to work with in Zynga. She’s one of the best people in this space because she’s very educational and has tons of experience. But coming back to your question – a lot of the work revolves around testing, getting the data, getting the numbers, backing up your thoughts and ideas and assumptions with the actual data, competitive analysis, understanding what people in this space are doing, and so on.

 It’s not about making things look pretty. It’s actually coming in with an idea of how we can do something in multiple steps that benefit the business, how we can prove that value and then what are the tools and partnerships we need to bring in. I agree, there are a lot of people in this space – some people might not be good, some might even be faking it.

But anyway, if I bring one person onboard, one way I would evaluate them immediately is check what their game plan is. Not the one thing they plan to do, but the end result. Because you need to be on a certain type of cadence. It’s the same with a UA manager. They won’t tell you, “I will just adjust the campaign budget on Facebook”. No, they will have a game plan. They will say, “I want to build a creative process.” Or “I want to build a process around geo expansion.” Or “I want to build a process around event optimization.” Right? There are multiple different layers within the best practices and methods that an experienced person would have. 

ASO is similar. They need to think three, four or five steps ahead, and they need to have an ultimate goal in mind, which is, “How do we drive conversion rate?” and “What do we do in order to get more people to actually install our app and engage with our business?”

Shamanth

That makes complete sense. And really when I say ASO experts, I think there are a lot of people that are somewhat hacky in nature and you’re right. I’m just calling out exactly what you’re looking for also, because I think ASO can come across as not precisely measurable as user acquisition. So it sounds like you’re really looking for rigor and thinking about what the end outcomes should be.

Let’s stay on the topic of hiring. You talked about your first couple of hires and how your team’s grown. What did you think about hiring agencies versus contractors versus full-time folks? And which of those is the right decision at different stages of your growth?

Simon

We were a startup when I started and we were at Round B. Now we’re a publicly traded company in a span of slightly less than two years. And we’ve done it as a very lean and very mean machine.

There are different ways of doing it, and no one size fits all. But with that being said, you really need to come into it with an open mind. I know some folks that say they want to build the team in-house and only in-house only. That’s the best way to guarantee expertise, they say. That’s all they’ve done and that’s all they’re ever going to do. And on the other hand, I know people who say they have no time, they like agencies because they don’t need to train anyone. They say it’s such a headache to interview people and bring them on board. I mean, interviewing one person is all right, but it’s a lot easier than trying to interview seven people for a period of six months!

So there’s different lines of thinking. But personally speaking, it’s easier to fill certain niche roles with contractors, and it’s faster because they move fast.

You don’t really need to have the approval of the entire team, for example. As opposed to hiring a full-time person, contractors are really like a nice band-aid. But even with contractors, I think long-term. If I’m hiring a contractor, can I potentially convert them into full-time? If I do convert them, will they stay with me for half a year? I don’t want to hire a person for a month – it’s just really not the best investment.

So even with contractors, think about what they can do for you long-term and whether or not you want to work with them for a year, let’s say. So contractors are the easiest solution.

Then I think full-time folks. My thinking for that is also different from a lot of people in the industry. When I hire full-time folks, my main question is whether they can be my right hand, in whatever the role is that I’m hiring for.

They might be junior, and it might take a year or two of working with me. But eventually, I try to see if I can safely tell them, “I want you to take over while I’m gone on vacation for two weeks.” Can they actually do exec reports? Can they do the functions that I do? Can they be my right hand? I like to see professional growth and I think people are motivated by that. So my full-timers are always very capable.

I do not like hiring people just to execute. Sometimes people say I’ll hire someone very junior so they can execute. If I’m hiring a person like that, they need to show me the potential of being my right hand in 2, 3 or 4 years, whatever that time period is.

And then lastly, agencies also play a big role. We have worked with agencies over the last couple of years, and we’re working with them now. We haven’t taken everything fully in-house, and we’ll see how things go.

Once you go public and you mature and have been around for 5, 6 or 7 years, you might build your own internal team. But a lot of companies in Round B or C stages work, and even some really famous companies, are still working with agencies and they’re still employing outside resources.

The reason is that agencies, especially the good ones, employ highly experienced media buyers. It’s really difficult, especially in this environment, to secure that number of experienced media buyers. Hiring 5 people with 8 years of experience will take you forever in this market. Those people are in demand!

So bringing in an agency is important. But whatever your role is in the company, if your job depends on the performance of this agency, such as mine, you must ensure that you have your own probation period, for the lack of a better word, for the people the agency gives you. Don’t have your junior media buyers overseeing an agency. You should be involved yourself to make sure—at least in the first couple of months when you’re ramping up— that the people that you’re hiring are top level from the agency side, and they’re as good as the agencies claim they are! Because you will trust them to run things for you, you’ll have them build the reporting and all those cadences, right? At least in the early stages, you should be involved.

Shamanth

Yes, I certainly think that the agency-client dynamic is particularly sensitive. One phenomenon I’ve seen playing out in the industry is that the person pitching is generally experienced and the person executing is junior. That’s one pitfall. And you touched on the other one, which is that the VP or the CEO hires the agency on the client side and then basically steps out of that relationship. And the junior person is left overseeing the agency. I think both of those are somewhat common pitfalls and to be avoided.

I’m also curious. You’ve talked about this from a UA perspective. Does the same line of reasoning apply for creative production as well vis-a-vis full time versus contractor versus agency?

Simon

It really depends on how labor-extensive your creative processes are and on your goals and needs. Creative can be a bit different. Your creative team can be catering to multiple stakeholders – not just UA, but also brand, your company’s product content, and so on. So from a UA perspective, I would say it’s similar. You need to have maybe a couple of core people in-house, but then you can outsource. Sometimes in-house teams cannot produce the assets if you have a lot, and oftentimes you do need to outsource. But you need to have quality checks in-house and you need to have people in-house who are putting things together and are a middleman between the agencies and yourself.

I don’t have a game plan or a playbook for that. But the core definitely needs to be in-house. You’ll need to have a couple of people controlling for the quality, and they can also relay the brand messaging, relay what works and what doesn’t. You need people that know and understand the product well in order to outsource and give the agency the expectations of what the creative work needs to be.

Sometimes the company can be very involved in your creative department. Let’s take Airbnb, that’s a good example. It’s a company with a very strong visual appeal and visual messaging, right? I’ve personally worked with someone who was the CMO at Airbnb, and they told me that 99% of the work was outsourced in the early stages because they had so much work being done on brand, on performance and a few other things.

So really, it depends on your company, but the quality controls should always be in-house and stay with you.

Shamanth

Yeah. Even if the execution is done externally, the quality control has to happen internally. I think that’s a useful distinction. And I’m curious, how is your team structured just now between functions?

Simon

There’s a dedicated creative team. Then there are producers, and there are content people. We have a clear separation. Then there’s the marketing team where we have product marketing with CRM, with all the internal initiatives where we’re trying to convert people and drive monetization. And then, on the UA side, specifically for my team, I have an ASO person.

I have a person that manages the agency relationship. I have an agency which has five or six media buyers. Then I also have my MarTech person. And lastly, I have my right hand, so to speak – a UA manager who looks after all the ad hoc requests and builds a lot of executive decks for me. And who just really drives better efficiency by really freeing up from day-to-day. That’s very important, and I want to emphasize that – you need to have someone on your team who pools up from day-to-day and who is not involved in launching creatives or overseeing any kind of relationship.

Once, I myself was in this position once, when I was reporting to someone on the UA team, and I was kind of an individual contributor. I always felt that my time was better spent looking at things where I had creative freedom to do something. I had free time to pitch in. Now I’m trying to adhere to that policy in my company. I have one guy who does a lot of reporting and ad hoc stuff, but that person has 50 to 60% of their time to focus on finding gaps in how we do things, and finding efficiencies. That’s very important and has been driving a lot of our progress.

I try to do that for myself too as much as I can, given that I run a pretty big team. I try to have some free time, so I can think about things and not just run around and execute.

Shamanth

True, you have to build slack in the system. Now, this is a bit of a U-turn. I remember you writing on LinkedIn—and I’m paraphrasing and probably butchering this—and you said, “Attribution is fine. It gives you a somewhat incomplete picture of what’s happening with your marketing. But if you really want a complete view of what’s going on, you really need to understand incrementality.” How does that play out in your day-to-day processes? I know you’ve talked about how you’re measuring SKAN and performance by making some estimations. What methodologies do you use? What tools do you use to understand incrementality?

Simon

I think you got the general idea right – you were paraphrasing pretty accurately!

It’s kind of one of the biggest challenges in the space. Firstly, it’s not really easy to measure incrementality, and secondly, a lot of people don’t really think about it this way. So it’s really a question of your fundamental thinking about what efficiency means to you at the marketing level. Why is incrementality important, versus merely having an attribution? And how do you go about measuring?

Let’s answer the first one – why is incrementality important? What’s the big deal? We’re already getting regular attribution, so why worry about incrementality?

Well, people come to your ads and to your channels with different intents. For example, one channel that has a very interesting relationship with attribution versus incrementality is Apple Search. Let’s say someone lands on your Apple Search page, after searching for Dave, or Zynga, Lyft, Uber, or Airbnb, or any consumer app really. That person comes with very high intent. And so what does Apple Search do, as in how does the mechanic work? Apple Search will then serve your own ad on top of your ad being shown.

Now, your consumer app already has an ad. So in essence, if a person who is looking for your app clicks on that ad, they’re going to convert based on something that they had a super high intent to begin with. What are the chances that they would convert if they never saw that ad and they’re already looking for the product? I think the chances are very high. So what is the real incrementality of that? As opposed to someone who’s never heard about your product, who is just coming in cold, seeing an ad and then converting.

So that’s a really good example. Why do we do that on Apple? Why do people even run ads?

Well, there’s always logic behind it. The reason is that if you don’t bid on your own space, someone else can bid on your churns. And then, one of your main competitors can be shown above you to a person who was looking for you. That person is now downloading your competitor’s app. Such an interesting dilemma, right? What about those people who would probably download your app, but instead, they downloaded a competitor’s app because that ad showed up one row above you?

You need to understand each channel’s incrementality. You need to start breaking it down into elements. First, look at your portfolio. What can you do? What are some of the strategies? My first strategy is funny because it’s the least scientific of them all, but it’s also the most accurate and the most brutish in a way. Start by turning your paid acquisition off and see what happens.

That is a very baseline strategy. And then start adding one channel at a time and see what happens to you overall. So for example, let’s say you have a dashboard which tells you that today you had 5,000 installs. You turn off all your paid acquisition and end up with 3,000 installs.

The second thing is to leverage the capabilities of the UIs. Unfortunately, that’s becoming more and more limited. Facebook used to be able to run an incrementality test, not so much with iOS 14, but you can still do it on Android. And it’s fairly accurate.

It’s not there on iOS. But you can maybe use Android results as a proxy for iOS. I would say a lot of times it’s fairly accurate.

Google has an incrementality tool too. You can actually run an incrementality test on Google. You can do it on Snapchat. Snapchat does not have it in the UI, but you can set it up with your reps. TikTok is getting very close to that point too. So you can leverage your partnerships in order to measure incrementality.

On the portfolio level, I would say the best way is still to turn on and turn off your incrementality on the channel level.

The other lever is called match market. And that’s a fairly interesting idea. You take one market, let’s say, Columbus, Ohio. You flood the market with ad spend for the specific channels. You already had your baseline before – in Columbus, Ohio, you were getting 200 installs. What happens now, when you spend more money? This approach works for non-direct response campaigns. So things like billboards, radio, and so on. It’s really hard to isolate for geo-specific campaigns on big platforms like Facebook and Google and get efficient results. So if you end up doing a match market test, you will select one region on Facebook and your costs will skyrocket as compared to your national campaigns, because Facebook just doesn’t have enough signal to optimize efficiently for people that are just in Columbus, Ohio. That’s the flaw with this type of system – it only works for non-DR or non-direct response ads. So non-digital ads for the most part.

So there are different strategies to mitigate and different strategies to measure. But I think the good old baseline approach is probably the best, especially if you’re in the early stage of your company and you can afford to turn on and turn off certain channels.

If you’re a publicly traded company and you want to measure incrementality by turning off all your paid acquisition, then there is a cost to be paid there, in cost and efficiencies. When you turn things back on, your campaigns lose all the history, right? So they have to reoptimize again, and you can lose some money there. You’ll need to ramp up because you don’t know how platforms will behave. But if you’re in the early stages and you can afford that strategy, add a new channel, turn it on, turn it off. See what happens with your overall mix.

That’s incrementality in a nutshell – what happens when you turn off one channel? Do you actually get more overall conversions?

Shamanth

And from what you’re saying, it often isn’t necessary to use elaborate statistical techniques? Of course, if you have the resources, you could.

Simon, I know we’ve gone longer than we had originally planned. Thank you so much for sharing all of your insights with us. I’ve certainly taken notes, and I think this is going to be incredibly instructive for everybody. But before we wrap up, can you tell folks how they can find out more about you and everything you do?

Simon

Thanks again for having me, Shamanth. It’s been a pleasure to reconnect.

You can find me through my LinkedIn at Simon Vishnevskiy. You can also look me up through my current company, Dave, I’m head of UA there.

As a bit of a last piece of self-promotion, I actually help startups and a few other companies, and I provide my own consulting services, also through my LinkedIn. I’ve worked with some very reputable companies in the past, and if anyone wants to ping me on LinkedIn on anything growth-related, I’ll be happy to answer the questions and consult. It’s a very dynamic space, things change a lot, and I am very, very thankful to have had great mentors and people that I can go to. And I just want to pay it forward and have this type of an impact on the space.

And this conversation with you was really great in that regard. Hopefully people can take away something from it. And if people can teach me and correct me, I’m open to listening and having this type of communication and feedback!

Shamanth

Wonderful, and you’re certainly paying it forward. We’ll share all your links in the show notes, and thank you once again, for being in the show.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify, Google Podcasts or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!