ABOUT: LinkedIn  | Glovo



ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

🎛How Georgy’s team collects all ASO KPIs in a single dashboard.

📸How to automate the customization of screenshots and metadata for multiple countries.

⚡️How seasonal offers and campaigns can be emphasized through app store screenshots.

💁🏻‍♀️Every app is different — why that makes automation of screenshot testing challenging.

1️⃣The most important metrics to look at on the ASO KPI dashboard

⚖️Filtering different countries to measure effect of featuring in each country individually 

😄Different ways in which the dashboard is extremely actionable 

🌤How you can track ratings on a day-to-day basis by using the dashboard

🏆How Georgy’s team uses repetitive testing to ensure his winners are reliable – because of the way Google Play measures confidence levels.

💸What are some of the ways organics are affected by paid or offline marketing?

🌏How Glovo does incrementality tests at a regional or city level.

🙌🏽Why Glovo’s team set up a dedicated automation team for paid acquisition.

KEY QUOTES

How to collect all the data in one dashboard

All information was mentioned in our ASO KPI dashboard, so from different tools like iTunes, iTunes Connect, Google Play Console, from different third party tools, like Apptweak or Sensor Tower or Appfollow as well, and try to pull all information in one landing page to increase the visibility of the process, to increase the efficiency of analysis that we are doing, and to automate the process.

Important KPIs

The most important metrics are first time downloads, conversion rates, and in the conversion rate, people who are on the app product page versus people who download the app. Of course, we also look at the app rating details to see what is the dynamic of an app rating on a daily basis to this dashboard as well. 

Why all the metrics should be in one place

There is the opportunity to look through on which places you were featured on the App Store and Google Play Store. It is also very interesting to look to because this gives an uplift in traffic as well, so you can get an idea why traffic increased immediately. You shouldn’t spend 1000 hours looking for different tools, to look at what you’re going to do.

There are insights in the ratings

You can interpret the ratings, or if something’s wrong with the ratings, or if you have a bug or something’s wrong, you can immediately look on a daily basis. You can see if you have many 1 star ratings, you can immediately react and inform your developers that something’s going wrong.

It is important to get to a confidence level

I’m trying to do repetitive tests, so if I do one test, and it’s successful after one-two months, I repeat the same test to make sure that this is a winner because the confidence rate inside Google Play Store is so low — it’s like 90%. If you know statistics, the lowest should be 90-95%. So 90% confidence is super low. To improve it, we are doing the repetitive test.

FULL TRANSCRIPT BELOW

Shamanth: I’m very excited to welcome George Natsvlishvili to the Mobile User Acquisition Show. Georgy, welcome back to the show.

George: Thank you for having me again. I’m super excited to be a part of this podcast, and I’m really happy that I can contribute.

Shamanth: I’m excited to have you because you have been a guest in the past, and definitely a lot of people received your last episode very well. The reason I also wanted to have you back was because of something you mentioned almost casually in the last episode, which is about automating a lot of ASO KPIs, which is not something that a lot of people talk about, or think about. So, I’m excited to dive into that with you today.

George: So there are different approaches for optimization of this automation. We can automate the analysis of the optimization through our special ad tech team, and you can automate screenshots or metadata through the special tools as well. 

So first of all we just collected all of the ASO KPIs — installs, first-time downloads, the different types of ASO KPIs that we need to analyze the processes before and after implementation, before and after keyword optimization, and then made a list. 

Then through our ad tech team we will try to connect all the tools into one landing page.

All information was mentioned in our ASO KPI dashboard, so from different tools like iTunes, iTunes Connect, Google Play Console, from different third party tools, like Apptweak or Sensor Tower or Appfollow as well, and try to pull all information in one landing page to increase the visibility of the process, to increase the efficiency of analysis that we are doing, and to automate the process. 

Right now we don’t need an additional person because everything is automated, and there is no manual work that we have to do. On the other hand, we want to move forward and ultimately make automations of screenshots and metadata because if you are across 30 or 40 countries it becomes super easy to upload the screenshots. There are special tools that can do this. There is a new novelty through iTunes API, you can do the screenshot and previews via automation — it’s I think live since one or two weeks. Through the iTunes API, it allows you to do automation of video uploading the videos and screenshots, but if you want to move forward with that, there is a special tool named Fastlane. 

Through this tool, you can actually automate the screenshots and metadata itself, so it’s very useful. It will take a very small amount of time versus to do it manually. It is the automation of uploading the screenshots.

Shamanth: So if you have, let’s say 20 countries, you put all the country screenshots in one single folder, and have the iTunes API exactly pull from it.

George: Exactly, with Google Play play and iTunes API there is a direct connection with iTunes and Google Play. You directly upload everything in one click versus to go through and upload it for each of the languages.

Shamanth: Right, And all of this is through fastlane?

George: Exactly 

Shamanth: Gotcha, and speaking of screenshots, is there any way that you make the process of testing screenshots more efficient and automated? 

George: In terms of making it more efficient, of course, because we are communicating with the branding teams, with the short notice of COVID, for some countries — we updated the screenshots — because during the COVID, there was a higher demand for the grocery category. 

So we switched and put more accent on the grocery category on our first three screenshots. In general, if we have some campaign, it’s better to also pop up on app stores and Google Search to ensure it is seen. Even through video or even through screenshots, it will help us to be transparent and to increase the current rates and to get a message to increase awareness of this campaign and other users.

Shamanth: Is there any way that tracking all of the App Store screenshot tests or knowing what is out in many countries, is there a way to automate that? Or is there a way that can be seen as more efficient?

George: On that I’m still working. That is a huge job, and you can’t say that one rule applies for all apps. Every app has a different approach and different screenshots, and different types of screenshots works for each app. It depends on the industry. Based on experience, I’ve worked with many apps, and you can say that this set of screenshots are going to work differently or 2-D or 3-D screenshots sometimes work. It depends on the industry.

Shamanth: Sure, and just to step back and talk about ASO KPI dashboard, what are the most important metrics that you will look at on a daily basis in the dashboard?

George: Of course,

the most important metrics are first time downloads, conversion rates, and in the conversion rate, people who are on the app product page versus people who download the app. Of course, we also look at the app rating details to see what is the dynamic of an app rating on a daily basis to this dashboard as well. 

Of course there is the opportunity to look through on which places you were featured on the App Store and Google Play Store. It is also very interesting to look to because this gives an uplift in traffic as well, so you can get an idea why traffic increased immediately. You shouldn’t spend 1000 hours looking for different tools, to look at what you’re going to do.

You just go to the one landing page and it shows everything. Also you can see the country — it is super quick switching the country and filtering, for instance, by app store segments as well. 

And of course the rankings: where is your ranking and what are they doing? How good are you? How are your competitors? Also, how many competitors appear on top of your branded search? 

Shamanth: That’s a lot of information, but it’s very important information that’s so scattered. I also liked that you said you have the countries filtered, so you can basically see this for different countries.

George: That’s correct. Also, you have a possibility to look through at what time, or with which release, you do updates of this before and after. It’s also very important to track these to not go to the different pages and see what you implement on the way. It’s nice to have everything on one page.

Shamanth: You did mention one example of how if you get featured, you will know that’s why it’s happening – and that’s why your organics went up. That’s a great example of how this dashboard can be so actionable. What are some of the other common ways in which all of this data is actionable that would have been so much more challenging without a single dashboard?

George: For instance, to play with app ratings, how to increase the app rating, how to improve your daily app rating, especially on Google Play Store, because it’s very complicated. People can’t just rate inside the app, we have to redirect them to the Google Play store page, and it’s super inconvenient to rate there versus iOS devices. Apples, three years ago, allowed you to get a rating inside the app, so it’s very convenient for users, and it’s becoming very convenient for us because we don’t lose the traffic versus these conversion rates from inside the app to the Google Play Store.

Shamanth: And you say, because you have this dashboard, you get to track the ratings on a day to day basis?

George: Exactly.

You can interpret the ratings, or if something’s wrong with the ratings, or if you have a bug or something’s wrong, you can immediately look on a daily basis. You can see if you have many 1 star ratings, you can immediately react and inform your developers that something’s going wrong.

Shamanth: Right, and if you see a big drop in ratings, you know that immediately rather than having to log into the rating dashboard and then look at it. Considering that Google Play Console lets you run screenshot tests, icon tests and so on, and I know you do run multiple screenshot tests so on, is that also something that gets tracked or do you just track that separately? How does that work?

George: I’m doing it separately.

I’m trying to do repetitive tests, so if I do one test, and it’s successful after one-two months, I repeat the same test to make sure that this is a winner because the confidence rate inside Google Play Store is so low — it’s like 90%. If you know statistics, the lowest should be 90-95%. So 90% confidence is super low. To improve it, we are doing the repetitive test.

Shamanth: What do you mean by the repetitive test?

George: We tested it again just to make sure that it is the winner. 

Shamanth: Speaking of looking at organic trends, you spoke about how it’s done. You’re getting featured. You can see that in the dashboard. Something else that can impact organics is paid marketing activity or even offline marketing activity. So you’re running a lot of print ads or outdoor ads…

George: If you are launching the TV campaign, it also is going to have a huge impact on your organic because people watch TV and then go to the App Store and download your app, so it’s not purely organic. It’s because of TV commercials. Also, some user acquisition campaigns might bring incremental traffic, but some of them even cannibalize your traffic as well. For instance, branding search traffic, they are cannibalizing the organic traffic of your brand — if it’s really high. 

Shamanth: Right, and some of these, for instance incrementals on paid or offline, do you get an understanding of that from your dashboards? How do you track or monitor that?

George: For that, we have a separate test of incrementality that we are doing per region or per country or per market or even per city and and per channel like Google or Facebook. Then we track the results, but it’s separate because it’s the kinds of tests. First of all, the App Store and Google Play Store can’t split by city. They only show per country, so if we did a test for the city, it’s useless to use any of them. 

Second, it is better to have the tests in one place — incremental tests, because here there are only organic dashboards and not paid acquisition.

Shamanth: I also like how you described the incrementality, and how you run it at a city level or at country level. If you’re doing it at the city level, how do you get city level installs of data?

George: And if you do per city, you are choosing multiple cities in the same region, you just choose the same city or similar cities with the same pattern. During the before test during the two weeks, what’s going on there is the same trend or not: then launch a test. During two weeks while the test is launching, you see the results, but after test results are also very important because two weeks is not enough. The effect will be longer than two weeks. Even if you have test on for two weeks, the effect and the impact of these two weeks might be longer.

Shamanth: That’s so interesting because A it’s not an exact science, and B it’s not something that a lot of teams and companies do with that diligence.

George: Correct. Actually, many companies are struggling to understand what is the actual incremental uplift from Facebook or Google AdWords for my organic. This is one of the key points that arise based on data, actually they are doing the budgets, they are calculating the budgets, and some of them just set their probability based on their opinion. They’re not even based on any data, so basically it’s a completely wrong approach.

Shamanth: This is so fascinating, Georgy. I’m curious because you guys have done so much to automate and add so much efficiency to your organic tracking and understanding your organics at such a deep level. Are there similar automation initiatives that you guys do or the paid UA that you can speak to?

George: Of course, we are doing it, but it is another team — a paid team. They are mostly focusing on operations stuff to automate, so they are more mostly doing manual jobs. We are trying to push them to do automation stuff for some part of acquisition. For couriers acquisition, we already implemented the automation process of automation bidding and also the automation of reports for other directions for users acquisition. We are in the process to implement automation. 

For some stuff, we already did a huge job. We even created this special sub team, a tech team, because it was almost impossible to reach our tech team to do this kind of stuff because it’s not only marketing, but there are all other teams that have a huge impact on our operation and our revenue. To do some small stuff on the user acquisition side is a lower priority than to fix the operation site. 

We have some challenges to do, so that’s why almost one year ago we created and developed an ad tech team. I’m also involved in a tech team because we did more than 100 automation implementations for different fields for API, dashboards, templates, and different tools like web hooks to connect data warehouses with different third party tools and dashboards. 

Because of this team, we also do special user acquisition such as game play, playing the game inside Facebook. We also do it through our tech team. We did a great job, and we are the only one among the 1 million apps that have a specific dedicated team for user acquisition automation. 

Moreover, this team has such a big success that other teams are also using their help — the grocery teams are also using the help to create dashboards and templates and to connect to do automation on their staff operations team. They are doing a huge job, and it already has an impressive impact across the organization. 

Shamanth: That’s very impressive. Perhaps this could be the topic of a subsequent episode, we can really dive into a lot of the other automation work you’re doing in much more detail. Georgy, perhaps this is a good place to wrap this episode. Thank you so much for being a repeat guest on the Mobile User Acquisition Show. Excited to put it out there.

George: Thank you so much for inviting me and super excited also to move forward and to talk to you about the tech team, the main challenges we faced, what we do, what we are doing to optimize the job and be more efficient in all directions. Thank you so much for inviting me.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!