ABOUT MIRELA: LinkedIn  | Twitter | Zinio

ABOUT ROCKETSHIP HQ: Website | How Things Grow | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

👎What caused Zinio’s iTunes rating to plummet to 2.3.

👀How Zinio identified the key issues that resulted in their users’ unhappiness. 

🚫Zinio decided to stop all paid marketing (less than 10% of visitors that landed on Zinio’s app page would have downloaded the app due to poor rating).

😃Zinio identified the “silent majority” of happy users and targeted them with prompts to rate the app. 

📈Zinio ran tests on a small portion of different user segments, and then ramped up the successful campaigns to the entire segment 

💪🏽What apps should do when they are going through a redesign so as to mitigate impact and minimize the possibility of negative backlash.

KEY QUOTES

Ratings can change dramatically overnight

So when we noticed that our average rating dropped so significantly, that’s when we realized we had a problem.So we basically started reading each review that was posted after the migration. We started sending out surveys and we managed this way to identify the most frequent topics that the customers were complaining about, which turned out to be some of the features that were left out from the old apps. For example, the ability to delete multiple publications at the same time or the ability to download multiple publications at the same time.

How ratings affected paid marketing decisions

So we basically completely stopped doing any paid marketing during this time as it would’ve been a waste of our money. We all know that ratings affect the conversion rate from impressions to installs. Less than 10% of visitors that landed on our app page would have downloaded our app because our rating was so bad.

Identifying happy customers from in-app actions

When is a Zinio user most likely to be very happy with their experience in the app? And to this question, we came up with several answers. At the moment, it could be right after they made a purchase or after they interacted with the content by bookmarking an article or sharing an article or after several consecutive sessions in a very short amount of time.

Concerted effort paid off

In a few months we were able to recover and even surpass the average rating pre migration. So pre migration, our average rating was 4.2, immediately after post migration we dropped to 2.1, within a few months increased to 2.3, and currently we are at 4.5 stars. And also our app store rating has also increased, and now we are in top 20 within our category.

User segmentation to solve the ratings problem

You looked at 20 segments that were likely to be pleased with the app experience, tested app rating prompts on a small segment of these, and really doubled down on what was working.

How to prepare for ratings after a redesign

I would say, first of all, they should brace for impact because it’s coming. I would definitely start by identifying the most popular features, sending a survey, finding as many channels as possible to communicate with your user base and your fans to alert them of the coming changes through all the possible channels: email, social, and in app.  Also, offer instant and easy access to customer support so users can get a good response. Offer as many non-public feedback channels as possible. Try to divert the negative feedback from the app store and prepare as many negative scenarios as possible.

FULL TRANSCRIPT BELOW: 

Shamanth: I’m very excited to welcome Mirela Cialai to the Mobile User Acquisition Show.

Mirela: I’m very happy to get involved – hopefully the information we’re going to provide today will be very useful for the listeners. So today I’m going to talk about our app. Zinio is the world’s first and largest digital newsstand. We have a portfolio of over 6,000 magazines in digital format, which are being distributed through our online newsstand and mobile apps for iOS and Android. To date we have delivered over half a billion magazine issues to customers from 200 countries. We have pretty much most of the top brands globally. I will give a couple of examples: National Geographic, Time, The Economies, GQ, Vogue and many, many others.

Shamanth: Yeah, and that’s precisely why we are here because you manage mobile marketing on a very, very humongous scale. And the specific part that I do want to dig into is a turn around that I think is very, very impressive. The Zinio app’s rating had dropped to 2.3, and you had to get it back up to 4.5. Right. And I think it’s such a cool and interesting and compelling story that I would love to dig into. Before we get to that, I would love for you to tell us why the app rating actually dropped to 2.3.

Mirela: Sure. So we were funded in 2001 which means we’ve been around for 18 years and through out this time we redesigned our website and apps several times. We all know that are vital for any user facing apps in order to stay fresh and current in a very fast changing market, but more often than not redesigns feel very unwelcomed, resulting in angry users who leave negative app reviews and give a low star ratings. 

This is actually what we went through last year when we had a complete platform upgrade and server migration, which resulted in a public backlash for several reasons – primarily because we left out some of the features that our existing users enjoyed a lot. So as a result, a significant number of our most loyal users started posting negative reviews in the app store and gave us one and two stars, which ultimately caused our average rating to drop in just a couple of months. I would also mention the fact that the lower ratings ended up affecting our app store ranking. Pre-upgrade we were in the top 40 within our category – and post upgrade, we ended up in the top 500.

Shamanth: That sounds scary to go through. Especially since your best users are basically having this mutiny – and certainly a complete revamp, a complete redesign can be pretty dramatic. It can be a very intense period. And now you’re having to deal with this customer backlash. How did you guys identify the sources of the problem? Actually you’re seeing all these reviews. How do you identify what the key issues were? 

Mirela: We are constantly monitoring our ratings, our reviews and constantly answering the reviews.

So when we noticed that our average rating dropped so significantly, that’s when we realized we had a problem.So we basically started reading each review that was posted after the migration. We started sending out surveys and we managed this way to identify the most frequent topics that the customers were complaining about, which turned out to be some of the features that were left out from the old apps. For example, the ability to delete multiple publications at the same time or the ability to download multiple publications at the same time.

So, I mean, these were our most loyal users. They’d been with us for years. So obviously in their libraries they had thousands of magazines. So when we removed that feature, well, the reaction was not exactly pleasant.

Shamanth: Out of curiosity, did this not come up in the research process before the redesign at all? 

Mirela: Unfortunately not. You know, I mean, it’s such a big project. As I mentioned, it was a redesign–so front end, back end, server migration, all the data had to be migrated. It was a huge amount of work. 

Shamanth: I can imagine that that level of complexity that goes into a redesign across, you know, hundreds of thousands of users, hundreds of magazines, I can imagine it’s easy to get some of these decisions off and that can very adversely impact the user experience in a way that you wouldn’t necessarily foresee. So your ratings dropped to 2.3 in a very short period of time, in a couple of months. And while the rating set down, how did your growth trajectory change? How did your paid marketing spend change while that rating was down? 

Mirela:

So we basically completely stopped doing any paid marketing during this time as it would’ve been a waste of our money. We all know that ratings affect the conversion rate from impressions to installs. Less than 10% of visitors that landed on our app page would have downloaded our app because our rating was so bad.

We knew that basically it would’ve been a waste of money getting users to our page and not getting any installs.

Shamanth: Right–so step one, we’re going to stop spending. You’re gonna stop getting new users. That wasn’t going to be enough because the rating was still at 2.3. What next? How did you address the problem of a 2.3 rating? 

Mirela: Well, first of all, once we were able to identify the main reasons customers were complaining, which are the features I mentioned, we added those to our roadmap, and our development team started working again, and they introduced those features as soon as they were able to do that.

We took care of the immediate problem. And then the second step, we were focusing on identifying the silent majority. That would be the satisfied users who would rarely leave ratings and reviews. Even though they are very happy with their experience, they’re happy with the app–they just don’t, you know, they don’t feel compelled to express that happiness on public channels. So we wanted to identify those people and sort of encourage them to write us. 

So we did several brainstorming sessions and we tried to come up with different scenarios on how to find those happy customers. So basically the question we were trying to answer was–

when is a Zinio user most likely to be very happy with their experience in the app? And to this question, we came up with several answers. At the moment, it could be right after they made a purchase or after they interacted with the content by bookmarking an article or sharing an article or after several consecutive sessions in a very short amount of time.

This could all be interpreted as signs that they were very pleased with the experience. So we create several audiences based on these specific events: purchase completed, article shared, etc. We identified up to, I think, 20 segments. And then in each of these segments, we run tests on a very small portion–I think 15% of each segment. And after 12 months, we looked at the results and chose the top three winning segments, which was where the audience delivered the highest percentage of five star ratings. And then for those three segments, we ramped up the campaigns to the entire audience–100% of the audience for each segment.

In a few months we were able to recover and even surpass the average rating pre migration. So pre migration, our average rating was 4.2, immediately after post migration we dropped to 2.1, within a few months increased to 2.3, and currently we are at 4.5 stars. And also our app store rating has also increased, and now we are in top 20 within our category.

Shamanth: I think that’s so impressive. Not just because of the magnitude of the results and the change, but also because of the very structured process you took to get that, right?

You looked at 20 segments that were likely to be pleased with the app experience, tested app rating prompts on a small segment of these, and really doubled down on what was working.

Out of curiosity, if you’re comfortable sharing, what were the top three segments? 

Mirela: From what I remember, the top one was the users that had just completed a purchase. Number two was users that had a high number of sessions within weeks, and I can’t remember the third one.

Shamanth: I know it was a while ago, that’s certainly impressive how scientifically and how methodically you approached this. And if you’re open to sharing, what did the prompt look like? You prompted these guys to rate. Was there anything around the messaging that you can recollect? 

Mirela: The message was very simple. We were basically asking would you rate Zinio five stars, and there were 2 options: yes and no. That’s it.

Shamanth: That is simple. That didn’t really require rocket science to get to that point. You needed to do a lot of analysis to understand how to really move the needle on these ratings. 

Mirela:  I should also mention that we actually tested several creatives on very small segments, and this was actually the winning message. The one that I mentioned–would you rate Zinio?

Shamanth: Right. What were some of the ones that did not win that may have surprised you?

Mirela: The other ones had a lot more copy in the creative. So, we said let’s “ok, let’s just reduce the copy.” Make them a very simple creative – put five stars in there and ask– “would you rate us five stars?” That’s it.

Shamanth: Keep it simple. Sometimes you don’t need to get very fancy. Yeah, and Mirela, what advice do you have for folks who are about to embark on a redesign and want to make sure they don’t go to 2.3 rating? 

Mirela:

I would say, first of all, they should brace for impact because it’s coming. I would definitely start by identifying the most popular features, sending a survey, finding as many channels as possible to communicate with your user base and your fans to alert them of the coming changes through all the possible channels: email, social, and in app.  Also, offer instant and easy access to customer support so users can get a good response. Offer as many non-public feedback channels as possible. Try to divert the negative feedback from the app store and prepare as many negative scenarios as possible.

You can imagine, maybe the password reset is not going to work or maybe the login did not recognize–and have a plan for each possible scenario.

Shamanth: Plan for the best. Prepare for the worst. Not everyone’s going to be able to bounce back from 2.3 to 4.5, but you did. Amazing for you, Mirela. Thank you for being on the Mobile User Acquisition Show and sharing your time with us today. 

Mirela: Oh no, thank you so much for the invite and always, always happy to get involved. Wonderful.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!