ABOUT GEORGE: LinkedIn  | Glovo

ABOUT ROCKETSHIP HQ: Website | How Things Grow | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

🤷🏻‍♀️ Is it better to attain a top rank for a less competitive keyword – or a low rank for a highly competitive keyword?

⚡️ How to measure the impact of keyword optimization.

📈 How improving ratings improve not just organic conversions but also performance of paid campaigns.

⬆️ How Glovo’s team improved ratings from 3.2 to 4.3 on Google Play

🤔 An example of an App Store screenshot that Georgy expected to see winning but didn’t win.

🛠 What tools Georgy recommends for app store conversion testing – and the problems he sees with third party tools.

⚖️ How testing can yield different results in the short term vs the long term

🔍 The ASO KPI dashboard that lets Glovo track and monitor their key organic metrics.

📝 Why Glovo’s team sometimes runs re-tests on their past successful tests.

😃 How testing the ratings prompt improved Glovo’s ratings significantly.

🙌🏽 Why Glovo does SEO activity to drive organic traction for their app.

📶 How Glovo’s organic growth team is structured

KEY QUOTES

How to approach keywords

It’s better to rank on a top one or top two positions on a lower competitive keyword and lower traffic versus to rank on 20th position on highly competitive keywords because you will be not visible and you’re going to lose your keyword space.

The direct impact of ratings on conversions

So, there is a huge correlation between conversion rate and ratings. So people, especially the browsers who browse and who are landing on your app page see and scan your app page very well and the main focus, they pay attention to your ratings.

The difference between 3.2 and 4.5

We increased conversion rate by 7%, but in total. Our traffic is very large, so if we improve even 7%, you can imagine that it brought approximately during the short period of less than one year more than 3 million marketing savings budget for us.

The cons of a third party testing tool

I am not super satisfied with this third party tool because they redirect it for the fake screen. So the user behavior for our visitors shows that some of them think that it’s connected to a bug inside the app. So we want to avoid this, and now we are focusing on testing on Google Play Store. 

Testing is not a one-time activity

We retest our successful tests — that’s the only way you can guess if the results are true or not. Also, we test before and after metrics, so we can get an idea.

Small changes can have huge impact

So we ask a person, as soon as he has a positive delivery, to rate our app. But the text that we showed on the popup message was super long, so no one wanted to bother reading the whole text. So we’d get a lot of “no” in this case. So, we just switched to a simple, one-line text, and it actually works much better. We increased by 0.2 in our rating because of this text change.

Unexpected ways to get incremental lift

Many apps that you see think it is enough to be on the App Store or Google Play Store or other app stores, and that this is enough traffic that they get. But they are missing out on a lot of opportunities because there is a lot of other incremental traffic from web users that don’t use the app or they don’t like to use these app stores.

The benefit of hyper specialists

Instead of hiring the new people and doing many of the jobs, we are focusing on efficiency and optimization. So, in terms of hiring an internal person, we have a highly qualified external person, so freelancers are working on ASO and SEO.

Automation is the key to cost efficiency

Everything is right here with one click, so instead of hiring a person to do this job, we ask our ad tech team to automate the process. For us, it’s very efficient, one of the most efficient teams in our company because we brought already more than 3 million marketing savings for the company with three people or four people working in my team. 

FULL TRANSCRIPT BELOW:

Shamanth: I’m very excited to welcome Georgy Natsvlishvili to the Mobile User Acquisition Show. Georgy, welcome to the show.

Georgy: Thanks so much for inviting me, Shamanth.

Shamanth: I’m excited to have you, especially because the organic side of mobile growth is something that not a lot of people talk about at least in the world that I work in, so I’m excited to dive into all of the possibilities therein. You do a lot of work in very different areas of organic growth. One of them, the more basic, is keyword optimization, which is one of the more low hanging fruit. Now, is that usually a one-time effort or is it ongoing, and can you speak to that process?

Georgy: Actually, keyword optimization is a part of app optimization for mobile and part of search engine optimization for websites, so it’s a constant process. When you put the proper keywords on in your title, subtitle, short description or keyword fill, you have to check how your app is ranked on App Store or Google Play Store. Sometimes these keywords are super competitive, so in most cases you will not get on the top position. You have to search for not high ranking and high-traffic and high competitive keywords, but mostly you have to find a combination between the most valuable keywords for your app and the less competitive one, and

it’s better to rank on a top one or top two positions on a lower competitive keyword and lower traffic versus to rank on 20th position on highly competitive keywords because you will be not visible and you’re going to lose your keyword space.

Shamanth: So how do you actually measure the impact of keyword optimization? Because you’re saying, look, the keyword ranking is improving, but how do you measure the impact of that on your growth?

Georgy: Usually we are tracking which position has worked for these keywords, so it should be known and grown keywords in most cases. We also track the organic traffic on the App Store and Google Play Store, how we change if organic traffic changes in that — because of these keywords, we get a lot of traffic. In Glovo’s case, it’s super complicated to track it this way, because we have a lot of other marketing channels such as offline, TV campaigns, so it’s very hard to see the results of keyword optimization. However, we can see as if we are top two, top three ranking, it means that the keyword is going to bring us traffic.

Shamanth: Yeah. Is there a way to isolate TV, out of home, etc., and say, look this actually happened through keywords, or is there no proper way to do that?

Georgy: The idea of it is to do an implementation of keywords, before and after, within the same atmosphere, so the same view. For instance, if you have a TV commercial before, we can say, “Okay, we have a TV commercial after, for one month, for instance, and we do implementation —  if something changed or not, and how fast.”

Shamanth: Right. 

Georgy: We also launched a couple of projects about incrementality, and we tried to find out what is the incremental impact on organic for all these channels, such as user acquisition and TV. However, it’s super complicated, and we need the time and space, say a couple of years or more, to see this is the seasonality, to see the exact trends, to define the regions, and to define the majority of the market as well.

Shamanth: Indeed, that’s one of the things I admire about your work because there’s no exact data available. If you’re doing a pre/post analysis, you have to make sure everything else is the same; and, like you said, you’re also looking at what is the actual incrementality from TV, out of home, and I think that’s actually very complex as you pointed out. Another aspect of your work has been around ratings, and I understand you had to improve ratings for the Glovo App from 3.2 to 4.5 or iTunes. To dive into that, is that true, that’s what my research tells me… ?

Georgy: So on the App Store we improved, but 4.2 to 4.7, but on Google Play Store we improved from 3.2 to 4.4. And it’s really, really good. We saw the results of increasing the ratings because of the dramatic increase in conversion rate because of them. Also, we get the opportunity to ask Google for a feature because if your app is lower than a 4.0 rating, no one is going to consider featuring you in the Play Store. So it’s a win-win situation for us – and from this, we got a lot of incremental downloads for us.

Shamanth: Yeah, and that’s a huge improvement. As you said, the conversion rate improved substantially – what have you seen as the relationship between rating and conversion rate? From 3.2, did you see the conversion rate improve as your rating went up or did it just go up at 3.9 to 4, what is it like?

Georgy: Actually you are right, conversion rating improved together with improving upon ratings.

So, there is a huge correlation between conversion rate and ratings. So people, especially the browsers who browse and who are landing on your app page see and scan your app page very well and the main focus, they pay attention to your ratings.

So if your rating is lower than 4, it’s very hard to convince them to download your app. Also ratings are visible not only for organic, for search on Google, for app page, but also for paid — one of the parts of App Store optimization that’s also visible for paid campaigns for Google AdWords for instance. So it brings not only incrementality among the organic users but also the paid users as well.

Shamanth: Yeah, right, and it’s applicable across the board for all your traffic. I’m curious, how much improvement could you expect to see if you went from 3.2 to 4.4-4.5?

Georgy:

We increased conversion rate by 7%, but in total. Our traffic is very large, so if we improve even 7%, you can imagine that it brought approximately during the short period of less than one year more than 3 million marketing savings budget for us.

So you can imagine how big the impact was from only a tweak in this improvement in app rating.

Shamanth: Yeah.

Georgy: Besides that, we did a lot of jobs in terms of testing different parts of app store optimization, such as video graphics, adding the videos, updating the screenshot that we can also see the huge impact from this implementation.

Shamanth: Yeah, and I understand you ran a lot of different tests on the App Store, right?

Georgy: Yeah.

Shamanth: Was there something that was surprising in all of these tests you ran, like you tried to test the graphics, you test videos, did you find that surprising?

Georgy: What’s surprising is that based on my experience, you should never take experts’ opinions inside the company because people’s opinions are different. At the end of the day, what we are thinking, maybe it’s something different versus what people are saying. Our customers are the key — even when we are testing super crazy ideas. One of the ideas that we were thinking that would be good based on my experiences sometimes get really good results among the customers, among the users because you should never only take into account our opinion inside the company, so we test, and at the end of the day, you will get very good results.

Shamanth: I can relate to that. I’m often very wrong — I run on the paid marketing side of things. Oftentimes if I’m very excited about an ad creative, it oftentimes just doesn’t work well, and something is looking terrible, and it just works really well. That often happens.

Georgy: It’s often happened even when we are thinking, “Okay, this is going to be based on creatives,” basically the best creative, but at the end of the day, it failed.

Shamanth: Is there something that comes to mind as something that surprised you — you didn’t expect this to win and at it won?

Georgy: Yeah, we put the screenshot that we were thinking would improve the conversion rate on the popular item in the screenshots. We changed the color of the background, within the proper background, we tested it, and unfortunately it failed.

Shamanth: Yeah.

Georgy: So we find another solution, we just change the background, we change the visuals, we change things, we put, instead of popular item, the popular category and hopefully it works.

Shamanth: Interesting, so even if you put a popular item, it wasn’t the best performing screenshot, right, with the popular…

Georgy: Exactly, because it’s also the seasonality factor, so sometimes the popular item is popular only in specific seasons or for specific countries. So, if one item is popular for one country, it doesn’t really work for other countries.

Shamanth: That is so fascinating. Do you do the conversion rate testing on Google Play Dev Console or do you use third-party tools – or both?

Georgy: Usually I am using the Google Play Testing. Also, I use the third-party tools for App Store Testing, but actually

I am not super satisfied with this third party tool because they have a fake, they redirect it for the fake screen, so the user behavior for our visitors, and some of them think that it’s connected to a bug inside the app. So we want to avoid this, and now we are focusing on testing on Google Play Store. 

Even if the possibility is super limited, and the digital thing is not ideal, and the confidence level is super low, it may make sense. So it’s very hard to say if this test is really good or not.

We retest our successful tests — only that way, you can guess if the results are true or not. Also, we are testing before and after metrics, and we can get an idea. 

Sometimes before and after also, we are taking, for instance, one week or two weeks, it doesn’t work. You have to give more time, and in the long-term period that was also surprising. In the long term, it also might work. So we have a lot of cases that in the short term, before and after one week, it doesn’t work, but if we took two months’ period or one month, it really works, and we can see the possible increase of conversion rate. 

Shamanth: How do you look at a two month cohort? Because the Google Play Dev Console doesn’t give you…

Georgy: Yeah, you can get to the data from the Google Play Console as well to see the first hand downwards and the product page views, so all information is available for use.

Shamanth: Understood.

Georgy: Also, we develop an ASO KPI dashboard with all ASO KPIs that are possible in this world for Google Play Store and for App Store — it’s a huge job that our ad tech team does. So we have a dashboard from third-party tools and also from Google Play Console. From the App Store, it’s one dashboard, they are showing everything in terms of App Store Optimization KPIs, and it’s really helpful because we can see it on a daily basis. Unfortunately, they are not daily from the stores, but at least we can see what’s going on, and we can immediately inform the tech team in the case something happens with products or whatever.

Shamanth: What would you say are the top KPIs in your dashboard — the most important one?

Georgy: Of course, the conversion rate, organic traffic, traffic among the browsers, these are the most relevant, also the ratings as well because ratings have a huge influence on conversion rate. So these are the top KPIs that we check.

Shamanth: Right, got it. Speaking of ratings, you said on Google Play you went from 3.2 to 4.4, what were some of the things you guys did to move that number?

Georgy: There are different approaches that we implemented, so we started answering to negative and positive reviews on the Google Play Store. Also, it helps to change the visitor’s mind and our users’ mind, and some of them switch from one star to five star, to positive. We also started to report the inappropriate reviews — either reviews that people are using swear words or something that’s not related within the app, so it also helps us to remove inappropriate reviews from Google Play Store. Of course the one with the biggest impact was we switched popup ratings that asked users to rate our app — we tried different combinations. Even slight changes inside the text can improve the rating a lot.

Shamanth: Do you have an example of the slight change that…

Georgy: Yeah, sure — we have very old-fashioned text that is super long.

So we ask a person as soon as he has a positive delivery to rate our app, but the text that we show, this popup message, was super long, so no one wants to bother to read the whole text. So we’d get a lot of no in this case. So, we just switched to the simple text with one-line text, and it actually works much better. We only increased because of this text change — 0.2 in our rating 

Shamanth: Wow, crazy — you don’t really expect that just from the text.

Georgy: Exactly. No one from the tech team expected this. We had a big fight with them to change something. Also what we did, we are doing this right now with this popup through our CRM side. If you have an initial popup, you have to, each time, ask the tech team to implement some changes, and it takes resources, and they are super busy with other stuff, so it’s very hard to get their attention. What we did was we switched from initial popup to CRM popup ratings through one of our tools. I don’t want to do advertisements for it now. It helps us because each time we want to do tests, we are super quick, so we can do same day testing.

Shamanth: That makes the testing process much, much faster.

Georgy: Exactly and easier.

Shamanth: Yeah, and something you guys do, that I don’t know of any other mobile apps doing, is SEO. So tell me about what inspired working on SEO projects as a mobile app.

Georgy: Actually,

many apps that you are seeing that this is enough to be on the App Store or Google Play Store or other app stores, and this is enough traffic that we can get, but they are missing a lot of opportunity because there is a lot of other incremental traffic from web users that don’t use the app or they don’t like to use this app.

They are using mobile phones and mobile versions and searching through Google. 

This is a lot of incremental downloads that our other apps are missing. We found this out, we tried right now to develop our website — our website is at the beginning of development, we didn’t have it two years ago — we just started from scratch. All our competitors actually started with a website, so they are super good, and they did a great job when they first developed their website and then the app. We are doing it differently — we have a really strong app and now we are developing on the web, and because of that, we get a lot of incremental installs coming from the web. Other apps also have to take into consideration that they are losing a lot of potential incremental downloads that might come from their website.

Shamanth: Interesting, so you just secure it on Google Search. They go to the website, and then they see a call to action that prompts them to download the app.

Georgy: Exactly, yes.

Shamanth: What sort of incrementals are you able to see, just via the SEO?

Georgy: It depends on the country also because for some countries, people are super mature with their mobile and the percentage of mobile is very high versus the percentage of app users. For some countries it’s not, but at least we get +10% of incremental traffic from their end. Imagine, our app is getting a lot of traffic, so +10% for us is like a huge result.

Shamanth: Yeah.

Georgy: Also, right now, our app website is not ideal, but imagine if we get the ideal website — that number will increase twice I think.

Shamanth: Yeah, that’s crazy, and…

Georgy: Also, in terms of mobile, the share of mobile search increasing on a daily basis and on a yearly basis as well, and actually we don’t know in terms of application — maybe, every day, maybe in future five years there will not be application, there will be something else on top, so people are going to switch on that. But in terms of mobile, Google has existed for more than 20 years, so this will not be changing at least for nearly 10 years I think.

Shamanth: Indeed, that makes so much sense, and it’s like I said, not a lot of people do that at all. My last question is you guys are also a fairly sophisticated team, just from what you describe, it sounds like it. Can you tell us about how your team is structured? 

Georgy: Yeah, so

instead of hiring the new people and doing many of the jobs, we are focusing on efficiency and optimization. So, in terms of hiring the internal person, we have a highly qualified external person, so freelancers that are working on ASO and SEO.

We have also one SEO person who is in-house, and because it’s a startup, startup life today you have a lot of money and tomorrow you don’t have money, and you have to fire people. In my case, it will not be the case, because we have super-efficient teams. 

We are only four people, and we have a lot of optimization stuff as I mentioned ASO KPI dashboard, once we report, everything is automated.

Everything is right here with one click, so instead of hiring a person to do this job, we ask our ad tech team to automate the process. For us, it’s very efficient, one of the most efficient teams in our company because we brought already more than 3 million marketing savings for the company with three people or four people working in my team. 

This is a very good achievement, and I think, in my opinion, even inside my company has to follow this trend, and has to follow this efficiency — in terms of hiring people for money or a job, and then get them frustrated or get fired. They have to focus on automation, and this is a key learning that I get from this process, and this is the key trend that I think you will keep in terms of the high competition of startups and in terms of the startups that can get a lot of funds right now with this competition as well. So I think this will be a key solution for that.

Shamanth: Indeed, and I think that’s also very interesting because on the paid side, I think there’s a fair amount of automation because Facebook and Google are driving it. On the organic, it’s not nearly as common, and I’m impressed about everything you described. Georgy, I want to be respectful of your time, this has been incredible, thank you so much for appearing…

Georgy: Thank you so much. Thank you for inviting me.

Shamanth: Thank you for being on the Mobile User Acquisition Show.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!