Notes:

Mobile Growth Lab
Doors are open to the third edition of our workshop series The Mobile Growth Lab, designed to help you break the shackles of ATT’s measurement and performance losses and win in a post-IDFA world. The workshop kicks off on the 26th of July, 2022.

We have some exciting sessions planned and also some awesome guest mentors – Matej, Thomas, Claire & Eric.

Sign up for the free pre-course email series here: https://mobilegrowthlab.com/mgl3/
Register for the workshop here: https://mobilegrowthlab.com/

We have a growing community of mobile marketers!

The Mobile Growth Lab Slack: A community that was a part of our workshop series – The Mobile Growth Lab, is now open to the general public. Join over 200 mobile marketers to discuss challenges and share your expertise. More details are available here: https://mobileuseracquisitionshow.com/slack/

If you’re ready to join the growing community, fill this form: https://forms.gle/cRCYM4gT1tdXgg6u5

ABOUT ERIC: LinkedIn | Heracles Media | Mobile Dev Memo  

ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

🌰 Eric’s transition from analytics to UA

🧃 Self-learning that happened through a blog that eventually opened doors

🤿 Common misconceptions between ASO & SEO

🕹 How UA on mobile different from other media

🛠 The biggest mistakes made in UA during the early days

👾 AppGratis that closed down after a policy change by Apple

🧉 Bot installs were definitely a thing back in the day

🧊 The evolution of apps in terms of being professional and how it impacts their presence on the App Store

 🐧 Working at Rovio on the Angry Birds franchise

 🐥How the Angry Birds crossover became a hit

 🔧 The kinds of challenges that companies will present to a consultant

 📌 Agamemnon and what made it a distinct Saas product in the UA industry

 ✂️ The low down on programmatic – why it’s challenging & how to make it work

 

KEY QUOTES

The early days of using analytics in advertising

The company I was working at, Digital Chocolate, actually had a really robust analytics system and it also had a pretty sophisticated range towards live ops and running analytics for these free to play games that were new in Western markets. But the marketing team had basically no insight into how to spend money. So they would launch games and then just buy Facebook ads against them without knowing whether those cohorts of users would be profitable or not.

The birth of Mobile Dev Memo

I left a pretty big company to a moderately sized company to a very, very small company. I could wither and die. I’m in Helsinki, Finland, no one knows who I am. I’m two years out of grad school. If this startup doesn’t work, my career could be terminated. I didn’t have any money. I was a $1,500 flight away from any family and support network. So in my mind, there was a possibility that if things go wrong here, I could be trapped. So I said, I’m going to start writing a blog as a plan B, or like a parachute.

The difference between SEO & ASO

There’s no real SEO on mobile because there’s no real PageRank. So there’s nobody that’s crawling all of mobile content and allowing for some kind of intent to be accommodated for in search. There’s search in the App Stores and on Google Play. But that doesn’t link into content that’s been evaluated for relevancy for those keywords.

ASO is not just keyword stuffing

 That’s not true search capability that doesn’t necessarily capture intent, unless the intent is very easily interpreted through just like a superficial search term. All that does is match specific keywords in a limited keyword space to a search term. So without that ability, discoverability is basically 100% reliant on ads. And that’s not true on the desktop.

How a policy change killed an app

 The CEO was on a long flight to the US. And so everything was fine. He gets on a plane, and he lands. He’s getting a million text messages, annoying emails – AppGratis shut down because Apple had killed the app. And so his company was just done at that point. 

How bot installs worked in the past

You’d buy all these bot installs, and it would drive you to number one top download, and then you get a bunch of free installs. And that doesn’t work at all anymore. You can still buy installs, but the benefit is pretty muted now of getting to top position. You could do that as a one off, but how do you productize that and make it systematic? How do you build it? How do you build a business around that?

Why solo developers find it difficult to survive in the market

It’s that things in Silicon Valley are so expensive, that the only way to hire people and build teams and build products is to be one of these big established companies with a ton of money in the bank, or equity that’s actually worth something. Otherwise, you just have to be so charismatic that you’re able to build a cult of personality and you can hire people on the basis of your interpersonal communication skills and your ability to sell them on a vision.

The experience on Angry Birds at Rovio

I heard from one network that they actually thought there was a bug in their dashboard software because there were insanely high click through rates for our game, which then resulted in really low CPI. That was a fun experience, the brand recognition and brand equity drove a lot of installs. But then we also capitalized on that to the greatest possible extent by spending a ton of money against that and just driving a huge number of installs. 

The best way to learn and develop UA skills

 If your first job is doing UA for Uber or one of these apps that just is part of the zeitgeist and everybody downloads that as soon as they see the app, they don’t have enough salt. But I think if you want to learn truly systematic UA, you’ve got to work on something that has zero brand equity, because that’s how you really learn about optimizing campaigns and incrementally improving click through rates and incrementally increasing the user base and focusing deeply on early stage retention, and then the final conversion and all that kind of stuff.

Exploring CPG is lucrative when the brand is doing well

That whole CPG business unit was extremely lucrative, while the brand was at that peak reach. Then it became a liability after that, and we lost a lot of money. So I think you can connect the commercial properties of something like that to the brand temperature. But if something’s extremely, extremely hot, then yeah, you could probably sell toys. If something’s kind of waning or cooling then you probably can’t. 

The best approach to build a lucrative game

The ideal approach is to have the product teams working really closely with the marketing teams and establish a feedback loop between them and making sure that we understand what type of users we can reach easily with this type of game. If those aren’t the right users, then we need to adjust the game to make sure that it appeals to the right users. 

How programmatic works

With programmatic it’s that you think about the kind of canonical programmatic use cases of retargeting, and so what are you doing, you don’t care about some group that a user belongs to because you know everything about that user, why are we targeting a user, the group behavior is totally irrelevant. I’m just looking at the behavior of that user. If they were really valuable users, I want to get them back in the product.

The efficiency of programmatic

The way that programmatic is really, really efficient, is you say, “look, all these bozos are bidding at some low level for this video game,” because on average, those users that come out of that video game don’t monetize well, but I know this specific user x, I’ve seen him in my apps, and I know that he spends a lot of money. Now when I see him in this app, I’m going to outbid all those guys, because in aggregate, the users of that app are not worth very much. But I know that that one specific user is part of our tail of that distribution. And he’s worth a lot.

There’s always free content to learn from

The great part about being alive right now is this infinite pool of content available for free from experts. Probably my first stop would have been fun to learn something new like YouTube just because people put out the highest quality content you can imagine. And it’s free.

FULL TRANSCRIPT BELOW

Shamanth

I’m very excited to welcome Eric Seufert. Eric, welcome to the show. 

Eric Seufert 

Thank you for having me. 

Shamanth Rao 

Absolutely. Early on in your career, you were working on business, intelligence and analytics. How did you make your way into growth and user acquisition and how easy was that transition?

Eric Seufert 

I started my career at Skype, as an analyst. I was not doing anything at all related to user acquisition or marketing or growth, or the kind of consumer oriented aspects of the product, I was effectively working for the private equity company that bought Skype and was doing a lot of reporting around software quality. I was working at Skype in Tallinn, Estonia, where Skype was actually founded. And I got really interested in the premium model from working at Skype and seeing how it was applied there in a superficial way. 

I saw that there was this whole new breed of games that was proliferating on Facebook that was using freemium in a much more robust way and that was all happening just across the bay in Helsinki, Finland. So I applied for a job at a gaming company that was doing free-to-play Facebook games as an analytics employee, and I got the job and I moved. In my first week, my boss left the company. So I got promoted to the head of analytics pretty quickly. One of the biggest problems that the company had was that we had a fairly sophisticated analytics infrastructure and the game teams had all the data that they needed to optimize the game, to tune the game and to make the game really appealing to the different segments of users. 

The company I was working at, Digital Chocolate, actually had a really robust analytics system and it also had a pretty sophisticated rage towards live ops and running analytics for these free to play games that were new in Western markets. But the marketing team had basically no insight into how to spend money. So they would launch games and then just buy Facebook ads against them without knowing whether those cohorts of users would be profitable or not. I started looking into what analytics I float to the marketing team to help them spend money in a systematic way. That meant to not just generate users, but to also generate profit. 

I started thinking about that methodology of the entire funnel from marketing, marketing assets all the way through into gameplay. I started researching how companies approach this concept of the LTV, the lifetime customer value created framework for measuring that, at this company. We launched that as a metric that was available in the dashboard. And then that piqued my interest in the marketing side of things. I left that company and joined a startup that was also in Helsinki that was doing a location based game called Shadow cities, and I joined them as the Head of Marketing.

Shamanth Rao 

I would underscore that seemed like a very improbable jump. But it took the fact that you were following your curiosity about freemium at Skype to your gaming company, Digital Chocolate, and you followed your curiosity around applying analytics to marketing and then you made that jump. As you expressed, you moved into marketing and user acquisition in 2012. This was just when mobile was emerging and exploding, and as you said, a lot of mobile games were leveraging freemium in very interesting ways. 

How did you educate yourself about user acquisition and marketing at the time? As someone who had come from a very analytical background?

Eric Seufert 

Well, the honest answer is that I started my blog. I think that the best way to learn something is to give yourself the task of writing about it. So I think if I pick some random topic I’m going to write about, I have to do a lot of research on that and then put that into a blog format, and then pare it down and distill that until it’s the most highly potent 500 word presentation of information, there’s a lot of work that goes into that. 

I started the blog. I joined the startup, and it was a scrappy, small startup, I think it was 10 people. Digital Chocolate was a pretty large company at that point.

I left a pretty big company to a moderately sized company to a very, very small company. I could wither and die. I’m in Helsinki, Finland, no one knows who I am. I’m two years out of grad school. If this startup doesn’t work, my career could be terminated. I didn’t have any money. I was a $1,500 flight away from any family and support network. So in my mind, there was a possibility that if things go wrong here, I could be trapped. So I said, I’m going to start writing a blog as a plan B, or like a parachute.

If everything goes wrong, this company fails. My CV looks terrible, because Digital Chocolate when I left wasn’t really doing that well. And I jumped to this startup in an uncertain stage of its life. So if everything went wrong, and my CV looks like crap, because I left a great job at Skype and had two failed experiences, then at least I’ve got this blog that showcases that I know something, and I can use that to potentially get another job.

Shamanth Rao

Seeing you where you are now, I think it’s easy to forget that you were beginning at one point. I think it’s pretty incredible, just looking at the story that you told at how you basically built yourself an insurance policy for your career that you did not need to exercise. But nonetheless, it’s been one of the most powerful parts of your personal brand, even today. That’s essentially how I got to know about you. 

Eric Seufert

Actually, I did have to exercise the insurance because the second startup actually did end up shattering. It was a great group of people and had a really ambitious vision. But startups sometimes don’t work out. And it was the blog that actually got me my next job, which was head of marketing at Google. I was visiting Berlin for some reason, and I had asked a connection of mine to connect me to people there, just someone and meet them and talk to them. So I went and met them and talked to them. 

I wrote this blog talking about these kinds of concepts. Maybe they found it interesting and they read it. And then when they were looking to make their marketing hire on the basis of seeing how I think they reached out and they hired me. I’m glad I had that because if I hadn’t I don’t know what I would have done today.

Shamanth Rao 

The blog opened doors for you when you didn’t quite anticipate. Eric, you started working on mobile user acquisition in its early days. What are some of the fundamental ways in which growth and user acquisition on mobile is fundamentally different compared to other digital media? 

Eric Seufert

The meaningful difference is friction, that whole journey through the App Store presents on desktop, you click a link and then you’re at the destination. There’s a single step to click. On mobile, of course, you have to click and then that’ll take you to the App Store or Google Play. 

You look at the app and you look at the screenshots, you look at the reviews, and then you choose to install or not. And then of course, there’s an install process. So the obvious difference from a UX point of view is just that added friction that happens after the click, and the whole opportunity for the user to say, now I changed my mind or, actually, after looking at screenshots, this isn’t really what I thought it was, which is potentially not a bad thing. But there’s added friction, which decreases the install rate. 

From an analytics perspective, there’s the added complexity of having to deal with an attribution partner, or just doing attribution on mobile, which is capturing the advertising ID or the device fingerprint at the click, and then trying to attribute that when they open the app store up. There’s a whole industry that was created to service that problem, you have to deal with a third party. And the data is not always 100% reliable. And the second big difference some people disagree with me on is that

There’s no real SEO on mobile because there’s no real PageRank. So there’s nobody that’s crawling all of mobile content and allowing for some kind of intent to be accommodated for in search. There’s search in the App Stores and on Google Play. But that doesn’t link into content that’s been evaluated for relevancy for those keywords.

So, if I do a search for a game, or if I do a search for a flashlight, I get results that match those keywords, because they are pretty superficial keywords. And they’re easy to semantically parse. But let’s say that if I do a search for bank peer to peer lending app that connects to my bank account, there’s app stores aren’t doing what Google does with PageRank with Google search, and they’re not digging through all the content in the app, and semantically parsing it and building relevancy scores, and mapping out the keywords, all they’re doing is just looking for those keywords in the App Store description, or the title. 

That’s not true search capability that doesn’t necessarily capture intent, unless the intent is very easily interpreted through just like a superficial search term. All that does is match specific keywords in a limited keyword space to a search term. So without that ability, discoverability is basically 100% reliant on ads. And that’s not true on the desktop.

So those are the two big differences. And I think it’s why it’s tough sometimes for people that are coming from desktop to become really productive on mobile, because they tend to focus too much on things like ASO. Whereas ASO is not the same thing as SEO. ASO to me is optimizing the App Store page for clicks. So that’s thinking about who’s coming here. How do I match the intent of people clicking an ad with what they’re seeing in the App Store page. 

So I don’t have the ability to create multiple landing pages on the App Store or Google Play, and then drive different ad creatives to those. So if I’m thinking about optimizing the App Store page, I really need to think about optimizing my ad creatives. What ad creative does the best job of reflecting what’s on the offer page, and inspiring clicks when the user gets there? App Store Optimization is not just keyword stuffing, I don’t have any ability to control what people on any given day search for now, in the short term from day to day. So if all I’m doing is keyword stuffing and trying to capture those keywords, then I have no ability to control the growth of my product. I’m just experiencing whatever this current keyword search is, and I’m trying to maximize my piece of that, but I have no way to actually increase that current, I have no way to increase that force.

Shamanth Rao

That’s right and as you’ve written about this, ASO has a ceiling on how much traction it can give you. I think this also explains why a lot of mobile relies on paid marketing. Whereas you could certainly build desktop brands based on content marketing, and SEO, which isn’t nearly as easy on mobile. So since you’ve seen mobile and mobile user acquisition for a long time now, what were some of the biggest mistakes that you saw people making in the early days of mobile in 2012-2013, when mobile was just about people figuring things out? What were some of the biggest mistakes that you saw people making?

Eric Seufert

First of all, a lot of people just raised too much money. You saw a ton of people raising these massive rounds of funding because mobile was new and exciting and it’s a paradigm shift. That’s what gets VCs excited because that’s when the real money is made when these kinds of tectonic shifts are happening and these paradigm shifts are happening. These new waves of technology adoption are happening and whole new categories get created. A whole new billion dollar industry or multi billion and trillion dollar industry gets created and companies are created and so money was getting thrown at stuff that was related to mobile. 

But sometimes you just raise too much money. Some companies went after obvious short term opportunities that didn’t really feel consistent with the way that the ecosystem was evolving. If you think about app proxies, they raised something like $20 million. And they got shuttered pretty aggressively by a policy change by Apple.

Shamanth Rao  

If I may ask, what happened to these? What was the policy change?

Eric Seufert 

Well, the app proxies were doing the app store within the app store. This is backed by 2014. That was a big marketing tactic back in the day. It was like a rewarded install program. But they didn’t do ads, it was just like the app itself. So it’s like a catalog, and it basically incentivizes install ad.  I’m just kind of blanking on the specific implementation now. But they did it in such a way that it looked like an app store within the app store.

The CEO was on a long flight to the US. And so everything was fine. He gets on a plane, and he lands. He’s getting a million text messages, annoying emails – AppGratis shut down because Apple had killed the app. And so his company was just done at that point. 

It was a standalone app. They had some existing number of users that could still use the app, and then they could still have their clients apps be promoted in the app, but no new users could download it. They pivoted into something else. But that company was basically just killed by a policy change overnight. Which underscores the need to not be 100% reliant on a platform’s whim. but you can always kind of intuit that that is not really how Apple probably wanted their ecosystem to be used.  It just felt like well, okay, you see an opportunity, but does that opportunity feel like perennial? Or does that feel like just something that was either a loophole or just has not been addressed yet, because no one has tried it, but will clearly not be tolerated. 

I think you saw those kinds of mistakes. Then there was a lot of focus back in like 13-14 on driving traffic in a way that clearly wasn’t sustainable. So a lot of people who bought installs into 2014-2015, people still use install forms to some extent, but it was just way more blatant back there.

You’d buy all these bot installs, and it would drive you to number one top download, and then you get a bunch of free installs. And that doesn’t work at all anymore. You can still buy installs, but the benefit is pretty muted now of getting to top position. You could do that as a one off, but how do you productize that and make it systematic? How do you build it? How do you build a business around that?

That is like a base hit. I’m trying to build an all star team. I don’t care about base hits. I took this baseball metaphor too far. Now, I don’t know what the championship game is called. But I want to win the championship game. I don’t want to get a base hit. So there’s that kind of stuff that made companies look like they had traction in the short term, but it was never going to be sustainable. And once those tactics were removed from the playbook, then those companies flunked.

Shamanth Rao 

And speaking of short term tactics, you talked about how people bought bot installs, which are fake installs. And then if they had a ton of installs happening in a short amount of time, they would get into the top 10 ranks. That was certainly a tactic that was relatively common at the time. Out of curiosity, what do you think would have stopped anyone from just doing it all the time? And just getting a lot of free installs, just by staying in the top 10? What sort of budget would it have taken somebody? And why wouldn’t anybody do that on an ongoing basis?

Eric Seufert 

Well, because there were diminishing returns from doing that. You do it once and you reach this whole new audience, you do it the second time, and you reach a much more limited scope. And so the more you keep doing it, the more you just keep reaching the same people. It wasn’t free, it could be kind of expensive. And then also there was a risk that Apple found out that you were doing it that could be on you,

Shamanth Rao

So these were short term fixes, as you said, these couldn’t build a sustainable business. As the App Store has evolved, you’ve written about this, mobile as a category has matured over the years. The way I look at the App Store, the odds of an indie game, or a hobbyist app, hitting the top charts, is very low now. Everything is very professional whereas even as recently as three to four years ago, you could see apps that were made by a solo developer, just kind of nice and neat, but just not a professional setup, could hit the top 10. That was a very real possibility. So the

professionalization of the App Store, so to speak, is that a bad thing? How do you look at that evolution?

Eric Seufert 

I would push back on the premise a little bit. You’re right, that the time to global phenomenon status is probably longer now. Because everyone has a smartphone, and it’s hard to penetrate. But you can. And also, there’s not that shock of virality, deeply penetrating to the public consciousness. But I think you can grow an app without having a gigantic balance sheet and you can grow it into a big business or at least a business that is on a relative basis big but you’re not going to do like a trivia crack. Now, I’ve been looking at an Instagram kind of growth trajectory before they got acquired. I don’t think you’ll see that just because that was like a category leader in a category that wasn’t very inventive. 

What they did was they innovated on concept, but every category now has really great abstinence, so it’s just more competitive to be the best. You’re not going to find a dormant category and build the best thing in that category now very easily. It’s just impossible. That the premise that you’ve got to be a big established company with a huge balance sheet to build something successful now, to me that feels like a Silicon Valley centric premise, or way of thinking, because out here, yeah, that’s true, because you can’t hire people, unless you’re paying a massive salary. 

There were a couple of indie game developers that got funded last year in this space in San Francisco. But how do you recruit someone? The only way you can recruit people is if you give them founder level equity. That’s the only way because there’s a chance to get to make a lot of money to have a massive win. I think that people take the hiring situation and employment situations in Silicon Valley and they project that out onto the state of the app economy, or the state of general consumer tech. But it’s not because things are so mature, that there’s no way to penetrate the market now. 

It’s that things in Silicon Valley are so expensive, that the only way to hire people and build teams and build products is to be one of these big established companies with a ton of money in the bank, or equity that’s actually worth something. Otherwise, you just have to be so charismatic that you’re able to build a cult of personality and you can hire people on the basis of your interpersonal communication skills and your ability to sell them on a vision.

If you look around, if you’re a top class engineer, and you’re getting an offer from Facebook, that’s like all in multiple hundreds of 1000s of dollars a year versus going to a start up where you know, you’re getting Robin cash and some equity grants aren’t even that great, even if you exit for huge amounts. 

Now, it’s a no brainer that you go with the Facebook offer, you got to eat at some point, you just got to cover your bases. And the cost of basics out here has gotten so insane that normal jobs can’t do it. But if you go to Europe, you find these small companies, small startups everywhere, and you’re able to live off of startup salary. So I think people don’t have that mentality when they go to Europe. Maybe I’m definitely generalizing here. And I haven’t lived in Europe for two years, maybe the atmosphere has changed. But we could do it on savings, we could do it on the side for no income, and you can make that work. But here you just can’t.

Shamanth Rao

Interesting. One way that has manifested is, a lot of companies are going increasingly remote. And I think that shows just as a result of the economics in the Bay Area. So anyway, you spoke a while ago about just hyper viral apps like Instagram or category leaders. You worked on one of those – Rovio on the Angry Birds franchise, I would say that was perhaps among the first mass market experiences on mobile. And you worked on the launch of Angry Birds. This was the sequel, six years after the original 3 billion downloads. So how did you live up to the success of such a monster hit? What were some of the goals that Rovio had, and you and your team had for the launch of the sequel?

Eric Seufert

With Angry Birds 2 it was interesting. We launched that game and I think we had a really thoughtful strategy on how to launch it. So first of all, there’s a lot of anxiety in the company before Angry Birds 2 launched around whether that brand still had the amount of power that it had at the peak. So we wanted to make sure that no matter what happened, we had to have a contingency for driving installs with paid media. So we put together a number of these different scenarios  because we couldn’t really do a soft launch. So the power of that was in Angry Birds 2. It was in the name. The power in what was going to generate interest was that, hey, this is the sequel.  They wanted just to test gameplay mechanics under a different name.  We weren’t really going to know what the level of interest was until we launched it globally. So we had a number of scenarios where we would launch campaigns or change budgets based on our six or 12 installs. 

So what we saw was A, there was still a ton of interest in the idea that this was the Angry Birds game sequel, and there was still a ton of appetite for Angry Birds and the brand was still very powerful. We saw a ton of installs, we saw a ton of earned media. But then we also just found that because we also didn’t know since we weren’t able to test any ads against the Angry Birds 2 name in the soft launch we saw that getting people to click was really easy. We ended up spending quite a bit of budget on paid UA, just as a result of seeing how low our CPMs were.

I remember I heard from one network that they actually thought there was a bug in their dashboard software because there were insanely high click through rates for our game, which then resulted in really low CPI. That was a fun experience, the brand recognition and brand equity drove a lot of installs. But then we also capitalized on that to the greatest possible extent by spending a ton of money against that and just driving a huge number of installs. 

In that first week, I think we did something like 25 million installs in the first week. It was just super exciting. Now I’d say that that’s not an awesome place to learn systematic UA.

If your first job is doing UA for Uber or one of these apps that just is part of the zeitgeist and everybody downloads that as soon as they see the app, they don’t have enough salt. But I think if you want to learn truly systematic UA, you’ve got to work on something that has zero brand equity, because that’s how you really learn about optimizing campaigns and incrementally improving click through rates and incrementally increasing the user base and focusing deeply on early stage retention, and then the final corrosion and all that kind of stuff.

I think it’s hard to learn how to do UA at that level of granularity. And with that level of demanding attention to detail and demanding attention to the underlying dynamics of the metrics and the conversion and monetization.

Shamanth Rao

Great Angry Birds is also one of the few franchises that has made the transition from mobile to offline. From the perspective of somebody on the outside, such as me, it sounds like that’s a very successful transition. There was merchandise, toys, cookbooks, food products, amusement parks, a lot of stuff. So what are some of the factors that contributed to that sort of crossover being successful for Angry Birds?

Eric Seufert 

Good question. I would point out that actually the toys and stuff,

That whole CPG business unit was extremely lucrative, while the brand was at that peak reach. Then it became a liability after that, and we lost a lot of money. So I think you can connect the commercial properties of something like that to the brand temperature. But if something’s extremely, extremely hot, then yeah, you could probably sell toys. If something’s kind of waning or cooling then you probably can’t. 

The hard part about doing all that CPG stuff is, you have to build a huge team. So you get a ton of overhead. As the brand cools, sales decrease, and then you’ve got to basically have to restructure. So if you can outsource all of that, it’s great. You’ve scaled up. But there’s ways to measure brand reach, there’s ways to measure brand power. I think having that as just a binary property, you either do or you don’t. You could just look at Google Trends. 

I think the crossover appeals. Angry Birds has worked really well as a movie and as a cartoon. There’s YouTube specials with more than a billion views in aggregate. The game has got appealing characters, it’s got broad appeal in terms of tonality and themes, the art style is fun, it’s jovial. That appeals to every single person or appeals to a huge percentage of the audience.

Shamanth Rao 

You left Rovio after a while to start your own consulting practice. You worked with around 30 companies, many of which had an earlier life as very viral games. What were some of the problems that these companies would come to you with? And what would your engagement with them look like?

Eric Seufert 

There were a range of problems. One was just how do we build our own analytics to do this at scale? So what I saw back then, and this is two to three years ago, was that a lot of people, their analytics stack was okay, they had an attribution solution. Maybe we’d have Amazon Redshift as a data collector and then we’ve got some sort of dashboard on top of that, or they didn’t even have their own data collection utility, they were just using an off the shelf one like Amplitude. 

But those tools aren’t built to do marketing, they are built to do product analytics. You see that there’s this very deep understanding of how people interact with the product, there’s almost like a total absence of knowledge of how they interact with the ads that lead them into the product. And so when you have that dissonance there, it’s really hard to grow a user base.  You can work really hard to try to adapt the product to the users that you are getting in without this way to impact the type that those users look like, that’s the least efficient way to work. 

The flip side of that, which is also inefficient is to work really hard to understand what kind of users do well on the product and push those types of users in to elicit, that’s less inefficient. It’s a little bit more efficient than

the previous approach. And then

The ideal approach is to have the product teams working really closely with the marketing teams and establish a feedback loop between them and making sure that we understand what type of users we can reach easily with this type of game. If those aren’t the right users, then we need to adjust the game to make sure that it appeals to the right users. 

As we bring the right users in, we need to understand what about them makes them the right users, adjust our marketing to accommodate that. Having this back and forth iterative improvement workflow between marketing and the game team, that’s the most efficient way to work. I saw a lot of that type of inefficiency, which is you’ve got the game and you’re just working really hard to make whatever users that come into it work. I mean, it’s a lot of work to actually iterate on. That’s why it’s so inefficient. 

So I saw a lot of this type A inefficiency. 

Then you’re just trying to say, Okay, well, the marketing team needs to get a clue. And we need to make sure that they’re just at least aware of what’s going on. And they can start thinking more systematically about how they acquire users. That’s actually a pretty easy thing. It’s the lowest hanging fruit. 

Type B inefficiency is when the marketing team is really efficient. The game team is getting whatever they want to build and the marketing team knows who they can reach. And they say, “Look, this is the game, we can reach these types of people, but it’s not going to be efficient or profitable.” And so that’s a tougher nut to crack because then you’re talking to product managers, instead of marketing people who generally are less receptive to prodding and helpful feedback or constructive criticism. So that’s a tougher problem to solve. 

Those types of engagements intended to go well, but I don’t think I had that many of those. And then in that ideal state, there were a lot of companies that knew that they were in a type B efficiency, and they wanted to get to type C ideal state. Those are fun engagements. Because then you get to help build out analytics, you get to help build out workflows, you get to educate people, you get to just see a lot more of that company. And also the feeling when you see a company kind of transform along that path, and learn. 

A lot of due diligence for VC funds and private equity funds that we’re going to do an investment in, they wanted to just get some  educated feedback on how Company X was doing marketing. Whether the game or whether the app looked commercially viable. And then I actually helped a couple of companies do recruiting. So that basically looked like helping them put together a job description, hiring the first UAE hire, you don’t even really know what to put in there, helping them do interviews, helping them vet candidates, putting together job tests that you could send out to people to screen. So that was okay. I’m not a recruiter, but those are two very distinct problems that a lot of companies face, and I was happy to get paid to help them do that. 

What’s cool about that is you can meet a lot of people. And so I met a lot of people that weren’t there yet. If the company was hiring for a mid level, you are a recruit, but they were really bright and really sharp. I could help them get jobs later. That was fulfilling for a couple reasons. One is, it’s nice to be able to do that. And then well, that’s just an easy way to recruit for the next client you’ve got, you can instantly email somebody and tell them about a new job and get them ready. 

Shamanth Rao

You touched upon how there wasn’t really a product that enables a marketer to get a deep understanding of how their marketing is performing. Whereas by contrast, there are a lot of products that give insight into how an app’s analytics is performing, having product and service performing. And you sought to fill in that gap by building the backend tech that you originally had started working on, that took the form of a SaaS product. Can you tell us about the SaaS product and how it worked and what made it distinct?

Eric Seufert

The name of the product was Agamemnon. The way it worked was people would input rough aggregate cohort data into a model, then that would give them some insight into how those cohorts evolved over time. They could also plug in some kind of speculative future UA against general average spend performance, and then that would produce cash flow projection. You could also produce what your DAU would be in 100 days or something. And that was like the 0.1 that I launched. The plan there was to actually integrate that into the company’s own data warehouses. 

The product actually got acquired by a mobile gaming company called Network and I joined Network and integrated that there. I was building the SaaS product for about a year and I launched it in an open beta, I think, nine months after I started working on it. The open beta actually had like 500 users. By the time I shut it down, it got acquired. We had a few people that used it as their central dashboard for marketing everyday, they would take all the data that they had, all these new aggregated metrics that they calculated from performance data, and then reinsert them or they would add them into this. 

It was a pretty manual process so I was pretty shocked people used it that way. That  made me realize that I should have launched an open beta much sooner just to see how people were using it so I could make it easier. I waited too long. And I had a product that I had built for my understanding of how I would use it, but it didn’t reflect the reality of how consumers interfaced with it.

Shamanth Rao 

But not before it touched 500 marketers which I think is huge. I know the mobile user acquisition space. So think back three years ago, I think that’s humongous progress for an open beta. And also not surprising because you were solving a very real problem. As you expressed, there really wasn’t any product that met a lot of the needs that these marketers were having. 

Eric, something else that you have worked on quite a bit is programmatic. That’s a term that’s thrown around quite a bit, talked about a lot. And yet, it also seems to be something that’s very hard to execute. And in one of the groups that we are both a part of you said, and I’m quoting you, “I can see that it’s pretty expensive to get it up and running. My fear with a 30k test budget is that it’s too low to actually produce performance. Programmatic requires a lot of experimentation to find the right combination of values.” 

So tell us about why it’s that hard to make work, that even a $20,000 budget isn’t enough to test it out and make it work. 

Eric Seufert 

Firstly, I will point out that the group that we both belong to is the Mobile Dev Memo Slack group, which people can join if they go to mobiledevmemo.com. It’s a great source of knowledge and insight. So I urge people to join that. 

Why is programmatic hard? A couple of reasons. One is that most UA infrastructure is set up to think about bid logic at the level of the source that might be like a specific campaign, or that might be just a bunch of users that are grouped around device type, geography, country and maybe some other factors. And so you’re looking at aggregate metrics across some reasonably sized population of users, and how they behave within the context of a distribution. You’re usually looking at the average behavior, right? 

Okay, well, I’m going to go and buy users from Facebook that are based in the US on Android, and that’s worth $5 to me, on average. If I spent $4.50, I should be getting 50 cents in profit right? And then I look at those cohorts, and I look at the way they degrade and then I just kind of re-update those assumptions. It’s kind of like the Bayesian system. And it’s looking at these big groups of people looking at a sample.

With programmatic it’s that you think about the kind of canonical programmatic use cases of retargeting, and so what are you doing, you don’t care about some group that a user belongs to because you know everything about that user, why are we targeting a user, the group behavior is totally irrelevant. I’m just looking at the behavior of that user. If they were really valuable users, I want to get them back in the product.

My assessment of how much they’re worth to me now is how much their expected value of their future contributions based on their own personal past contributions. If I’m modeling that I might bring in behavior of other users, but it wouldn’t be the entire group. It’d be users that look very, very specifically like them. And those users might not belong to the same geo group. They may not belong to the same device type group, they might not belong to the same acquisition source channel group. It’s just that profile of users. 

If I’m doing a big network campaign, I can’t target people like that, that have some sort of profile, right? So then when I’m targeting, I’m looking at that specific user and I’m trying to estimate how they will continue to perform in the app, if I bring it back in, and I’m sure it’s worked for me. If I’m doing programmatic at scale for acquisition, I’m still looking for users. I’m looking at the user level, because in this campaign, if I try to run it on a channel that accommodates programmatic, it doesn’t do any sort of optimization like Facebook or Unity does, it just lets me buy it and lets me bid on a specific impression. 

If I’m not, then I either have to build all that logic internally, which is really tough. No one’s replicated what Facebook has done with campaign optimization. First of all, it’d be almost impossible to do that, because you don’t have as much data as Facebook does, you don’t have all these different pieces of data, all these different features of specific groups of users to make those kinds of assessments with. 

The network has done it because they just see so many users, and they see how they interact with so many different types of games, they can do way more complex optimization than you could because you only see your game, or maybe potentially a couple of games, but not that many, like a couple 100 million MAU. Facebook would see a billion DAU.  So you just can’t. So then what you end up doing is you’re trying to find a specific user and you’re pricing them on the basis of this group behavior.

That doesn’t really work just for that reason, because you don’t have the ability to optimize that bid with any kind of bid logic, you’re just using this aggregate user behavior for people that look like that user on the basis of these different groups that you’re able to apply to them. 

So the way to make programmatic work is to actually do the same thing that you do with programmatic in retargeting, which is create an expected value of that specific user, but  apriori, any sort of information about them in the total absence of data about how that user behaves, you’re trying to price that specific user now that works really well, If you’ve got a portfolio of apps, and you see 100 million MAU, or you see 30 million DAU, or whatever, and you’ve got a bunch of different apps, and you’re saying, “Well, okay, this user looks like this.” and when I’m trying to launch app B, and I’m going to try to find that specific user and put them into B, and I just go out and I do programmatic that way. 

But it doesn’t work if you’ve got a limited overview of users, and you’re not able to find them in other apps.

So the way that programmatic is really, really efficient, is you say, “look, all these bozos are bidding at some low level for this video game,” because on average, those users that come out of that video game don’t monetize well, but I know this specific user x, I’ve seen him in my apps, and I know that he spends a lot of money. Now when I see him in this app, I’m going to outbid all those guys, because in aggregate, the users of that app are not worth very much. But I know that that one specific user is part of our tail of that distribution. And he’s worth a lot.

Instead of trying to find that user in the expensive game where everybody bids really high, I’m going to try to find that user in the cheap game where everybody bids really low, outbid them all, but still at a price that’s way below the monetization that I expect from that specific user. And I’m going to get them that way. 

That’s how you get programmatic to work. So why is it expensive? Well, you need to see all those users; 

1. You need to have the analytics infrastructure to be able to break all that data down at the single user level, and do an automated way where you can just push that out to the exchanges.  
2. You just need to have oversight of a large number of users. And so how do you do that? Well, you spend a lot of money seeing advertising identifiers. 

I think to start a programmatic endeavor from scratch, first of all, there’s tech that needs to be built. And second of all, you just need a large data set. And if you already have a large data set, you still need to build the tech, if you want to do it right. It’s about half a million I think to build something like that. To have it be operational at scale to be acquiring a lot of users and to be profitable, I think it’s a huge investment. It’s a huge undertaking. 

Now, that said,  that thing is basically like a cash printing machine, because you’re seeing all this scale in places that other people can’t see, and they don’t have access to them. And now you’ve got companies like Beeswax that are giving access to all these different exchanges in DSPs, without needing to buy a seed. So you can get access to that inventory, these reps can be cheapest, it’s a valuable service, you can get access to the inventory without having to make the upfront investment in the seed. The minimum spend requirements for some of the seeds are really high, too. It’s really powerful. But there’s a huge amount of upfront costs that you need to be comfortable with before you start down that path. And I think the way that people mess that up, is that they try to do it cheaply, and they try to cut corners. And then you’re basically just doing a network style acquisition in a totally different environment where that doesn’t work. If you try to take the same approach that you have to buying users on Unity and you apply that on Exchange or DSP it’s not going to work.

Shamanth Rao 

Thank you for demystifying that. I really appreciate that clarity you provided because I don’t think programmatic is particularly well understood. For reasons you did explain, it’s very hard to execute. You’re basically building up in the absence of user profiles and building up profiles with a blank slate. That is an incredible explanation of why that is the case. Eric, this has been such a wide ranging conversation, and I’ve just learned so much. I’m always astonished when I read your blog, and your responses as to how in depth they are, how detailed they are. So I’m curious, who do you learn from? What are some of the sources that you look to, to inform yourself and educate yourself?

Eric Seufert 

Learning about and understanding programmatic was something that was a priority for me in 2018 because I came into 2018, not knowing anything about it. So I read basically all I could read on adexchanger.com, the CEO of Beeswax has put out a lot of good content on Twitter, Ari Paparo. I followed him on Twitter, and I watched a lot of his YouTube videos. A lot of these people came to Google and desktop and so that’s how a lot of this infrastructure has been built and I had no background in that at all. That’s how I’ve learned about that. I started reading a lot more about behavioral psychology and behavioral economics. I think that those are topics that are really applicable to anyone who’s playing with monetization in the product specially like a really big product. 

I’ve been deep diving more into data science recently and there’s a ton of good content available for free. There’s a blog called Towards Data Science you can consume hours and hours of content there and not even scratch the surface. I’ve been trying to dig in a lot of these topics probably since 2018 as well just because I want to add some dimensionality to my skill set and my professional profile and so I read a lot every day, try to set an hour for just reading and learning something every day.

The great part about being alive right now is this infinite pool of content available for free from experts. Probably my first stop would have been fun to learn something new like YouTube just because people put out the highest quality content you can imagine. And it’s free.

Shamanth Rao

I’m so amazed even more so than I was when I began researching you for this interview. And I’m just so astonished at just how repeatedly you got into a field and spaces that you had no background at all, from Skype to gaming, to your work in analytics to working in marketing. And then of course, building a product of your own. And now, everything you’ve told me about Programmatic and how you dived in and learned and figured everything, by yourself. I think everything you just said in this interview is so inspirational, so thankful for having you on the show. 

And I think this is perhaps a good time for us to start to wrap up. Before we do that, can you tell our listeners how they can find out more about you and everything you do?

Eric Seufert

I have a website mobiledevmemo.com. I have a book on freemium economics that I wrote available on Amazon, you can follow me on Twitter, it’s at eric_seufert. You could follow mobile dev memo on Twitter @mobiledevmemo, and also on Facebook. 

Shamanth Rao

Excellent. And we will link to all of that in the show notes and the transcript. Eric, very, very honored to have you on the show. Thank you so much for being here.

Eric Seufert 

Yeah, sure. Thank you so much for having me.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify, Google Podcasts or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!