💀 Welp! Our revenues dropped 75% – and they were all organic – with Sharath Kowligi(Director of Ad Monetization at GameHouse) 📉

What do you do when your revenues fall off a cliff? Today, we get to the bottom of things – and point to a way out.

I’m joined by Sharath Kowligi, Director of Ad Monetization at GameHouse and advisor to RocketShip HQ. This deep dive is based on a discussion that Sharath and I had with the founder of an app sometime ago(we’re keeping the app’s identity anonymous – although the learnings and takeaways from this discussion are widely applicable).

In this deep dive, we will discuss some of the specific factors at play that caused this app’s revenue to drop 75%, and what we see as the ways forward to help them recover ground. While it’s easy to think organics are hard to impact, this is a masterclass in how they can be moved and impacted, especially when they are central to an app’s financial future.

KEY HIGHLIGHTS

😵 The immediate issue with refunds that the app was facing.

🤷‍♂️ What happens when an app’s rating goes below 4.

🤑 Why paid advertising isn’t always the answer if organics drop on account of ratings.

🤭 What Google’s bad behavior thresholds are.

🔢 What metrics Sharath recommends looking at to diagnose organic drops.

📋 Benchmark organic to paid ratios for Android and iOS.

😲 How Apple perceives ‘bad behavior’ metrics as compared to Google.

🔓 Oftentimes just playing by Google’s rules unlocks a ton of organic traffic.

KEY QUOTES

Paid advertising is not a magic spell

Paid advertising is probably going to be useful as if you don’t see an automatic recovery of all organic traffic at this point after these particular issues with the refund, the particular issues with the temporarily low 3.6 reviews – once that’s sorted out, rating doesn’t automatically go up. At that point, I would say, okay, we don’t have any major crashes. We’ve solved our refund issues, we’ve solved the review problem because that was also temporary related to some other bugs. And now everything is cool.

The most important component of UA for an app is the app

The main things to look at are ANRs and crash rates, because that’s just basic hygiene, you want to make sure that the software doesn’t crash in any app, whether it’s a lifestyle app or game, what have you. It’s, at the end of the day, it’s software and if the software crashes, no one can use it.

How to source content ideas

I will also look at something like an aggregator for customer comments. And I recall that, when we started doing a lot of copywriting campaigns, it makes sense to look at reviews to use users’ words and do your ads, that also it’s really worthwhile to just look at a word cloud of all user reviews.

Diagnose not-so-great metrics on Android

if you’re organic to paid ratio isn’t well over one only if you’re buying more paid traffic than having organic traffic in Android, I would say, something’s up because it does a fair job. And on iOS, I would say it’s probably good to have a one to one ratio, because if you have a good crash free application that does what it’s supposed to, and what I mean by that, if it’s a free app, then does it have good engagement? And if it’s a paid application, does it monetize well enough? User reviews? I would say both of those are good, broad metrics to look at.

Apple metrics can be somewhat opaque

Unfortunately, Apple doesn’t give you the kind of insight in terms of – exactly here is where bad behavior threshold lies and point B is exactly where you are. And this is a much of an improvement you need to make. I would really enjoy having those kinds of metrics on Apple.

FULL TRANSCRIPT BELOW

Shamanth: I’m very excited to welcome Sharath Kowligi to the Mobile User Acquisition Show – Sharath, welcome.

Sharath: Hey, Shamanth, it’s good to be on the interwebs again with you. Cool. Let’s dive right into it..

Shamanth: Right. So sure you’ve seen the data. And, again, for transparency, we got to keep the app anonymized because we don’t want to disclose details, but certainly the dynamics and learnings are all real. Okay. So when we say that revenue dropped like 75%. This is not a super early stage app. They have fairly significant scale. But it’s not like going from $100 to $25. But all of those installs are for organic, did a little bit of paid acquisition in the past, but that was not on Android, they did a lot of it on iOS. And yes, it’s easy to say, “oh, you built your castle on Google’s platform. And now a ha, consequences.” It’s easy to say that. Go ahead Sharath, yeah!

Sharath: Right. I mean, like this particular app, the kind of users these are diehard users. It’s not, you know, typically, an entertainment app. It’s not a fad or any of those types of situations. Yeah, and they were doing quite well. And of course, there’s a little bit of concern always when you’re fully reliant on organic traffic because you don’t really know where that’s coming from. I think where Google especially has become incredibly transparent over the last two, I think, maybe three years really, like since 2017, they’ve been really, really good about telling developers and product managers and really anybody who takes the time to look at the developer console as to what they think of that particular app.

And when we dug into that even further, it became really clear that the issue actually happened with refunds. Where, for whatever reason, a number of paying users, an abnormally high number of paying users were demanding refunds. And they weren’t necessarily really leaving negative reviews. They weren’t necessarily experiencing crashes, nothing that they care to share about. You know, they were quite happy to trigger the refund issue. And the entire refund cycle that was really an outsize number of refunds at that point.

At the moment, we, I think, start looking at that and it becomes pretty clear that even though you see something like a 4.1 rating in the dev console, you actually do see at 3.6 rating for the customers. And then you see this issue with the refunds all coming together.

And then then you see this drop off, and it’s obviously hurting traffic, but I think what they also really cared about was it was hurting word of mouth, it was hurting perception of the app itself and in all the forums that people kind of talk about and and here’s where just logging in as a user shows you clearly, like it showed as when we were looking at it that, you know, this is a 3.6. And that’s, that’s totally different from what we thought. And there isn’t a hard and fast rule of what is a great rating and what is not a great rating, but a lot of public literature out there that shows that, you know, folks under 4 tend not to see a lot of like, under four stars out of five tend not to see a lot of organic traffic.

And I’ve certainly seen the reverse happen where applications that were either incredibly punishing to free users, or, you know, had their own sort of bug issues go from a three point something to a four or something, and then just have an insane amount of organic traffic. And versus its most likely true in this case there. Yeah, user ratings have dropped.

Shamanth: Yeah, yeah. And, you know, I think there’s so much that we can unpack in there. But it sounds like you know, even though this is a great app, there were specific things that are happening. For instance, the high refund rate that really triggered Google’s algorithm to say, right, let’s give them less organics. And that really built up over time to result in a 75% drop in revenues. Is that a fair way of assessing this?

Sharath: I think so. I mean, this is always the fun part, right? Because you now have a blackbox algorithm. And then you have significant things changing within the way the KPIs are assessed by the algorithm. How it weights things and and and when it’s weighted, that stuff is always going to be hard to determine. But it’s also obvious when you look at the final result, like as much as you’re never going to know what the algorithms are unless you actually see them or someone publishes them. It’s straightforward to understand what the impact is, because there weren’t a lot of issues with the application a few months ago.

And suddenly, things fall off a cliff, literally. And we have two big things that have changed. But the biggest one being the refund rate. It’s safe to say that this is as likely as it can get before you know, somebody who actually designed the algorithms come in & say this is what it actually is. One of the things that I really liked about the way that you approached this was, and of course, we both were on the same side of user acquisition, and now, you run user acquisition for a number of folks, a lot of it is paid traffic.

The go to answer for paid traffic is, oh, well, organic is uncontrollable. And like you said, you know, you’ve built your app on Google’s invisible logic. And so now you have to kind of deal with it. But the answer is more paid acquisition, and that’ll solve everything. I think that would have been like the obvious default answer to give in this case. And I really liked how you were like, wait a minute, this is just bizarre.

And, you know, let’s all just get extra sets of eyes. Let’s take a look at everything. Make sure that we’re not missing anything particularly weird. Just triggering that because, I would say that, like if I was to have a situation like this, the answer from I think most paid UA channels would be well, this is a great opportunity to increase your paid advertising budget.

And I think that’s, that’s a good time to be a bit careful because, I think it’s inevitable in any case where you have to start this spark with paid traffic. If you don’t have a lot of earned media or if you don’t already have a great following, that goes without saying, and I certainly agree with your thoughts on that. But when you have a situation like this more advertising is not always the answer.

Now then, let’s say it’s been 14 days, and we don’t see any improvement in organic, at that point, I would say, okay, restart paid traffic, but in a good market, don’t do random untargeted campaigns, no matter how much the four cent installs tempt you, I don’t think that that will change anything because you’ll probably get quality traffic at that point. But I would do that because in a tier one market and do a small paid campaign just to see if seeding the traffic and getting more positive signals out there to the Google machine or the Apple machine, if that makes a difference, but only after these issues are fixed.

And I think one of the things that I really liked about your approach to user acquisition in general is that there’s a support function that is very critical, very unlikely that anyone’s going to succeed hugely in the current state of the mobile market without paid acquisition, but if the product itself doesn’t have – hasnt cleared the basic metrics according to the platform providers, then more paid immediately is not the answer, then it’s much better to go back to product, fix the product, and then do paid acquisition.

Because ultimately you want your paid acquisition to scale. You want to get value out of the dollar. And then to decide for small teams whether paid or organic – all of these things are, these are difficult challenges. And the other one, there are millions of apps now at one point it was less, but now there are millions of apps.

Shamanth: And I think it’d be easy to say – spend more. And you’re right, but that isn’t always the answer. What I also like about your approach is how you clearly looked at the KPIs, even though organic can appear to be somewhat opaque, it’s not. And have you looked at very different metrics, very different KPIs that said, “look, this could be a problem.“

Can you speak to some of the other KPIs that you saw, you know, for instance, somebody else that watches this, or reads articles that we record this might say, “well, what are some of the other KPIs I should be looking for?” And I also like that Google likes to call this like the bad behavior threshold – like admonishing a child. So what’s bad behavior for Google?

Sharath: Oh, bad behavior for Google, I think. Yeah, this is pretty much, I got a good laugh out of it when I saw for the first time the bad behavior threshold. It started becoming less funny when I was looking through it.

At the time I was doing UA for a number of applications, when the first of these types of dashboards came in. And it was funny the first time that I saw it, by the time the 25th app that I was also looking at and I was like, “Oh, well, we’re probably in trouble at this point, because it’s this big red alert that we end up seeing.”

It makes sense for the platform providers to surface the application. And when you think about it a little bit from their perspective, it makes tonnes of sense. And I think the main issues that I would look at is usually the overview in the crash rates and also look at trends, and I would start there. The next place that I would look at is to try to understand what’s going on, with either customer support if you have the customer support team going or people talking about a specific issue.

And then

I will also look at something like an aggregator for customer comments. And I recall that, you know, when we started doing a lot of copywriting campaigns, it makes sense to look at reviews to use users’ words and do your ads, that also it’s really worthwhile to just look at a word cloud of all user reviews.

And, you know, I’ve often neglected that sometimes, because I was like, yeah, that’s what people say. I would much rather look at what people do. But sometimes, if all you’re seeing is dwindling organics, dwindling revenue, it’s also worthwhile to just look at all user reviews, put them into a word cloud, there are a number of tools out there that do that, some paid. I think that’s very valuable.

And then, of course, the last thing that I look at is really revenue. And that’s where we found a lot of potential issues with this particular app to look at specific user packs, is it a currency issue as it is the fact that you know, it’s not, it’s not surfacing? And then just recently, it again seems really bizarre, but revenue drops off, crash metrics are fine, no major issues with refunds. But, you know, stuff is broken somewhere. And quite often it’s also just at the store, if people can see the store.

Yeah, you’ve got your answer right there. And it’s important to go through each of these in order because you want to rule out the obvious stuff first, before you start digging in to the trickier things or the weirder things that are very particular to, you know, one specific application or one specific class of product.

And usually this is one of the big things and in Android, I would say,

if you’re organic to paid ratio isn’t well over one only if you’re buying more paid traffic than having organic traffic in Android, I would say, something’s up because it does a fair job. And on iOS, I would say it’s probably good to have like a one to one ratio, because if you have a good crash free application that does what it’s supposed to, and what I mean by that, if it’s a free app, then, you know, does it have good engagement? And if it’s a paid application, does it monetize well enough? User reviews? I would say both of those are good, broad metrics to look at.

In Google, you have a whole bunch of these dashboards and I would just go in order.

Shamanth: Yeah. So this particular app we looked at with had issues only on Android. Yeah, I assume those exact same bad behavior metrics happen on iOS. Low ratings, high crashes, refund rate, but the impact be just as much? Less than that? Not at all?

Sharath: Well, I think there would definitely be an impact, This is much probably not because iOS doesn’t normally give you that measure of organic traffic. But if you’re one of the lucky few that gets a whole bunch of iOS organic traffic, because you delight your users and Apple recognises it as such – and these things happen to you. Then you’re definitely going to see a drop like if you’re going to see a sudden drop in reviews, an increase in crashes, a high refund rate.

Especially all of those things happening almost right next to each other or adjacent, then you are definitely going to see a drop in revenue.

But having said that, Apple has also given a whole bunch of new tools for people to look at. And, I would imagine that eventually, the platforms do compete against each other. They compete for developer talent. They want the best, most dedicated developers, I think, on their platform, and, and so I’m sure that they would, there will eventually be parity and there has been for a long time. One gets better at helping developers understand their users and then the other follows eventually, if not immediately, but eventually.

Shamanth: Apple is a rather different beast. And it certainly makes sense and tends to behave according to certain slightly different rules. Certainly, I think what I took away from our last call, then and of course now is Google offers so much more visibility than is obvious to even some relatively savvy developers that I know. But they don’t seem quite as aware of the kind of metrics, the kind of visibility that Google can offer. And my encouragement to folks who, when confronted with organic issues or just even just looking for opportunities or even looking at the Google dashboards, looking at these metrics – try to make sure they are on the good right side of the good behavior threshold. And clearly, I think all you need to do to get a lot of organic sounds like is just to play by the rules.

Sharath: Play by the rules and when you do play by the rules, we’ve both seen what happens. There are definitely applications and I can count at least three, just off the top of my head that crossed well over 5 million downloads just on Android, on one platform. If you’re a developer, who, as much as you know, we want everyone to do paid user acquisition for millions of dollars, there are millions of developers out there who don’t have the budget, but playing by the rules, having a good product, having that validated by, you know, a platform like Google – good things will happen.

That I’m quite convinced of – I’ve seen this happen so many times in the last couple of years, especially, good application, low crashes, low refunds, high reviews, because you’ve done the right things. Yeah. And then you end up with millions of downloads, it is possible doesn’t happen to everybody, obviously. But in a product like this, I have no doubt that they will continue to see a good increase in their organic traffic.

Shamanth: Yes, sir. I think that’s a good takeaway, and certainly something for myself and all of us to remember to look forward. Sharath, as always, this is very insightful. incredibly helpful. Thank you for taking the time to share your wisdom with us.

Sharath: Happy to help, yes.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!