Note: We have a growing community of mobile marketers!

 The Mobile Growth Lab Slack: A community that was a part of our workshop series – The Mobile Growth Lab, is now open to the general public. Join over 200 mobile marketers to discuss challenges and share your expertise. More details are available here: https://mobileuseracquisitionshow.com/slack/

If you’re ready to join the growing community, fill this form: https://forms.gle/cRCYM4gT1tdXgg6u5

ABOUT GINA: LinkedIn | Latitud

ABOUT ROCKETSHIP HQ: Website | LinkedIn  | Twitter | YouTube

KEY HIGHLIGHTS

🌰 The meeting with Barack Obama

🧃 Launching Tumblr in Brazil & other parts of Latin America

🤿 The Global Ginga 

🚵 Getting hired by Duolingo as a consultant

🎧 Building the Duolingo brand

🪘 What were the first priorities to grow the brand?

🎰 Staying away from paid advertising initially

🚊 How did different markets across the world react to Duolingo?

⛩ Challenges faced in foreign countries

🚀 How did Tim Ferriss impact the brand

🎢 The transition from PR to growth

🎡 How the team picked experiments to apply to Duolingo

🕹 How Duolingo’s gamification came about

🛠 Adapting the product to suit international markets

KEY QUOTES

On having been picked to present to Barack Obama

I honestly didn’t think that President Obama was going to be there. I thought that we were just going to be presenting to people from the White House. I thought it was a cool thing anyway, because I hadn’t really been to the White House outside of the little tour that you do when you’re a kid. So it was a big surprise for me. And the other funny thing is that they put us in the little room all the way in the back. And I remember I turned to our CEO and co-founder Luis and I said, “Oh, we’re in the losers room”. 

But actually there were about five startups that they had selected, to present to the President in person. We were one of them. And it was a huge surprise to learn that because they didn’t actually tell us about what was happening until about 10 minutes before he entered. They were being really finicky about the angle of our poster and if our pitch was good enough, I just thought they were being super annoying. But then I understood later why they were being so meticulous and I’m glad that they were.

The beginnings of Gina’s career

I went to my career counselor and I remember she looked at my resume and said, “What have you done with your whole life?” 

It was so sad because I was a super nerd and overachiever. I cried. And I just couldn’t believe that nothing that I had done seemed to be worth anything to her. So I went, and I decided to work at a neuroscience lab for a year because I was interested in neuroscience. And I thought that maybe that would give me a more serious angle to things. And I don’t know, it would sound really impressive. I co-authored a study, but I didn’t want to work underground for the rest of my life. So I knew that that wasn’t for me. And then at that point, I applied to 100 different jobs when I graduated. I didn’t hear back from almost any of them.

Moving to Duolingo fulltime

They asked me to  take over growth. This was not an official thing to Duolingo, yet. They had  a product team that was doing AB testing. That’s when my team came in. We looked at all of the sub metrics that were never really paid all that much attention to. We wanted to try and optimize not just grow the day one but also the day 14, the 30. The plan was to look at referral, our activation, take a closer look at emails and notifications and take the company to the next level.

The inspiration behind joining Duolingo

I’m not American, and I would never have gotten all the opportunities that I had, had I not learned English at a very young age. And learning a second language, like learning English in developing countries can double or triple your income potential. I felt like I was in the top tiny percentage of people in Brazil, who would ever get the chance to do that. Just because I came from a good family. I believe that by investing in that I could help change millions of people’s lives

Using the right hooks to get journalists interested

I thought to email them and tell them that I’m in touch with this amazing guy who created the captcha and did all this stuff. I feel like Luis at the time was kind of trying to hide it because he didn’t want to talk about the captcha anymore. He was so over it. It’s been like 10 years already. 

I knew that the nerds and tech would totally geek out on the fact that they could meet the guy who had invented that. I worked with them with a team to figure out how to position the mission, how to position what he was doing, and it wasn’t too hard because I was sold on it. I really believed it. I reached out to all these different journalists and I helped Luis. I set up a lot of speaking opportunities for Luis in Brazil.

The decision to stay away from paid marketing initially

Not paid. We had agreed not to do paid from the very beginning, because we were not profitable. We didn’t have revenue and venture backing. We didn’t want to start paying for users until we knew that our retention was where we wanted it to be. We knew that we had an LTV for our users, which is finally the case now. So we started doing paid acquisition actually, quite recently, but we were doing other things other than the PR

Why too many competitors won’t work for this model/h4>

Well, fundamentally, it’s a group buying model. So you’ve got a sales team, you’re calling merchants, and at the end of the day, the point is that you’re driving customers to the merchants. So if Groupon drives 100 people and LivingSocial drives 50 people and then everybody else drives a few people, it’s not going to work.

Getting blocked by the government in China

When we launched, we got a million downloads the first day, which is huge. Then our app got blocked by the government. There was nothing I could do about it.  I had been in touch with Apple, and they were like, What is wrong with your app? And I was like, it’s not on our end, but they wouldn’t really believe us. And then  all the people who had downloaded the app started rating it like one star because the app didn’t work. So it was just so frustrating, after all that work to see that happen and not be able to do anything about it.

An experiment that worked out well/h4>

I remember the day our engineer decided to add a badge to our Android app, a little badge that says that there is something new on the app, and you have to click little red dots to find out what it is. He wanted to do that. I thought it seemed kind of spammy but I told him to try it. He just went back to his desk and did it anyway. It was like 10 minutes of running code, seven lines of code. And that was one of our most impactful experiments ever.  It was really a lot of experimentation, and reading and trying things out and thinking about, like, what are the most impactful places that we can touch? And what are the highest ROI things that we can work on?

Recognizng experiments that failed

I remember when we decided to redesign the newsletter. Anything that involves design was so much work.  I think on one hand, it’s because I couldn’t do it myself. I can write copy, but I can’t design. I need to rely on other people. The design standards are extremely high. In order to get something approved, it takes a very long time. We did all this work, and then it had zero impact on anything.

An experiment that may have impacted many other experiments

Our biggest failures were when we ran experiments, and they either worked or didn’t work. And then later we realized that actually, we weren’t tracking metrics properly. That, to me, is the most frustrating thing for the whole team. For example, for six months  we were working on sign-up/ sign-in things in terms of  optimizations, and then we realized that when someone who already had an account, but forgot that they had an account created a new account, the system wasn’t linking accounts. So they were creating new accounts that weren’t really new accounts, and this was inflating our number of new users and potential. We realized that we didn’t know how to fix this. We also did not know how many experiments this affected.

The intention behind gamifying Duolingo

Duolingo was originally launched with the intent of being a game. It was baked into the product from the very start before I was at Duolingo. The idea behind that is that learning a language is very hard, because it takes a very long time. It’s not the kind of skill that you can dedicate a weekend to and then be done. I studied Chinese for three years, and I still barely get by. So you really need to dedicate years of your life to this. Also binge studying one day a week doesn’t work, you need to do it very regularly.  Add that with the fact that now we’re telling people, Hey, you can do this by yourself, no, teachers are gonna be making you do it. Your parents are not gonna be angry. If you don’t do it. You’re not paying money, so you’re not gonna feel guilty. And it also competes with your Facebook time

How gamification helped Duolingo get to where it is today

People are like, yes, I want to take Harvard classes, then the retention is really low. Because it’s so hard to convince yourself to keep coming back. And to complete a course that you don’t lose anything if you don’t complete. The thought process was well, but people do spend all their extra time playing games like Candy Crush at the time, Clash Royale when they’re in line, when they’re waiting for a meeting with their lunch. That’s what they’re doing with their time. What can we do to make that be Duolingo?  From the very beginning, it was designed to be like a game, but it really took all these years. And I think it’s still a process to improve, because gamification isn’t just something that you throw onto a product. It’s adding layers and layers of things that make it more fun, like the points system and how you can  pass your friends, and you can unlock things. Now there’s a mascot and you can earn points and you can earn digital currency. Now you can buy things with digital currency, and you can get badges.

Going international with localized features

Our answer, I think, is very controversial here. Yes, I agree. People have very different expectations of what an app should look like & feel like. We chose to ignore almost all of that. And so what we did is we localized, obviously, so when we launched the app in a new country, we would basically just make it available in the language of that country. And then we would  launch an English course, for example, for speakers of that language.

Challenges faced in international markets like India

So when you buy your phone, it’s in English. And it’s just how it is. And also, typing in Hindi is supposedly very difficult. So a lot of people get these phones, and they’re in English, they don’t actually speak English. They just kind of learn how to use a phone. And then what happens is Duolingo notices what your UI languages and it adapts to that. So people were now seeing Duolingo in English, and they didn’t see the option to learn English, because from English, we don’t teach English, we would teach French or German. Our English from Hindi was never seen by anyone in India, we had to change the flow there specifically to match that.

FULL TRANSCRIPT BELOW

Shamanth

Gina, welcome! I would love to begin on the fourth of August 2015. You were in Washington, DC. What took you there?

Gina

I don’t think it was a very important day. On that date, I thought to myself, this is the peak of my career and I am going to quit. I was in the White House. And I had the honor of presenting to the former President Obama.

Shamanth

Wonderful. How did that happen?

Gina

I would love to say, President Obama finally realized that I am one of the coolest people on Earth, so he invited me to come and hang out. But that’s not what happened. They were having their first and what might honestly be their last demo day at the White House. And they selected  around 40 startups to go to the White House to present and we were one of them.

I honestly didn’t think that President Obama was going to be there. I thought that we were just going to be presenting to people from the White House. I thought it was a cool thing anyway, because I hadn’t really been to the White House outside of the little tour that you do when you’re a kid. So it was a big surprise for me. And the other funny thing is that they put us in the little room all the way in the back. And I remember I turned to our CEO and co-founder Luis and I said, “Oh, we’re in the losers room”. 

But actually there were about five startups that they had selected, to present to the President in person. We were one of them. And it was a huge surprise to learn that because they didn’t actually tell us about what was happening until about 10 minutes before he entered. They were being really finicky about the angle of our poster and if our pitch was good enough, I just thought they were being super annoying. But then I understood later why they were being so meticulous and I’m glad that they were.

Shamanth

I’m sure all that finickiness would have been absolutely worth it. That rendezvous with Obama was such a long way from where you were at in your own life. Even just a few years earlier. You graduated in philosophy from the same college that Steve Jobs actually went to? What was the path that led you from there from a liberal arts degree from a philosophy degree to becoming a growth leader?

Gina

My path has not been at all conventional or  this straight path. And also I didn’t graduate from Reed I actually dropped out from Reed just like Steve Jobs did.

Shamanth

I didn’t know that.

Gina

I transferred to Brandeis University, and I graduated from there. So I do have a degree in philosophy. And then I honestly didn’t really know what I wanted to do with my life, I thought that I wanted to be a film director. But

I went to my career counselor and I remember she looked at my resume and said, “What have you done with your whole life?” 

It was so sad because I was a super nerd and overachiever. I cried. And I just couldn’t believe that nothing that I had done seemed to be worth anything to her. So I went, and I decided to work at a neuroscience lab for a year because I was interested in neuroscience. And I thought that maybe that would give me a more serious angle to things. And I don’t know, it would sound really impressive. I co-authored a study, but I didn’t want to work underground for the rest of my life. So I knew that that wasn’t for me. And then at that point, I applied to 100 different jobs when I graduated. I didn’t hear back from almost any of them. 

I finally got an internship with a digital media agency. I realized that studying philosophy, you learn so much, and yet you learn nothing. I had no actual real world skills like how to use Excel. So it was kind of a really scary moment. And all I wanted was a job that would sponsor my visa to stay in the US because I’m Brazilian. So I took it and I ended up working in digital media.  

I learned tons and had to Google a lot on the job. I remember every day, I’d have to be googling things like HTML, HTML for Dummies, because I was afraid that if I asked people questions, and they realized I didn’t know what I was doing, I would get fired, and I would lose my visa.

Shamanth 

Yeah, the visa is a bit of a nightmare, and you and I have talked about what it’s like. So that’s how you got into the digital space. You learnt through a lot of Googling, you ended up in Tumblr. You basically launched Tumblr in Latin America. How did you do this?

Gina 

After the digital media stuff, I decided that corporate life wasn’t for me. I actually went back to Brazil, and decided not to do this anymore. Tumblr came and found me and it was really a chance moment that happened, because I think I’d done a very good job networking without realizing it before. I just met a lot of people in New York and really tried to do my best job, even though I didn’t like the work that I was doing. And I think that left a good impression in their minds. 

They came to me and said, “We need help with Brazil. We don’t have a budget, can you help us?” And I honestly had no idea. Tumblr was already available online. And there were users in Brazil, but the launch is the same thing that we did with Duolingo. It was more of a marketing play – a big splashy launch. We were going to get PR around this and get people talking about it, even though it was not really a launch if something was already there before. But it worked out. The only thing they asked me to do was to throw two big parties in Brazil to mark the launch of Tumblr. There were two 1000-people parties. This is probably the most stressful thing I’ve ever done in my life, event planning. 

The cool thing was that I didn’t spend time curating: all the artists, musicians & personalities that were on Tumblr in Brazil came and really represented what Tumblr felt like, in real life. It was  drawing upon all the theater experience I had in school, and I got to do that in the real world. I reached out to a lot of reporters, and told them about this really cool thing happening. There’s these Americans in tech who are super important, asked them if they would be interested in talking to them and found a way to pitch to all these different verticals that ended up being super relevant. And we got tons of attention and tons of PR, which worked really well. 

Also, it was about figuring out what was different about Brazilian usage of the product compared to that of the US. Turns out, people were not really logging or creating accounts and logging into Tumblr in Brazil, like they were in the US, they didn’t even realize that they were looking at tumblrs, they were just sharing links with each other to funny Internet pages. Like on Facebook Messenger. We started thinking that if we want people to create an account and spend time on Tumblr, we need to figure out what’s going to make them do that. We went after the content that really mattered to them and tried to get that on Tumblr. It ended up being a lot of comedy, a lot of YouTube celebrity type people, and then important media, corporations and museums. And I would go and pitch and say,  I think you guys should make a Tumblr, this is what I think you should do. I’m going to get one started for you. 

I knew from my agency experience that they wouldn’t want to leave an empty page up so they would have to continue it. So we got tons of new content, and then relevant local content. Did some community work also through a meetup that was their biggest ever with 1500 people. Then we did work in Chile, in Argentina, following Brazil, that was along the same lines, but  slightly different for each country. So for Chile, we ended up focusing a lot more on the music industry. And we also had meetings with the government in the Casa de la Moneda. They were interested in communicating with young people in their population more effectively. So we pitched Tumblr as a tool to do that.

Shamanth 

That’s an amazing story. You connected with a lot of the key cultural influencers, and told the key movers and shakers in the media. You just brought everyone together, and built this flywheel that started spinning. That’s such an amazing story. What took you from there to Duolingo?

Gina

I spent a year doing that for Tumblr, and then they were gearing up for acquisition. I didn’t know of course, they laid off my boss. And then they laid off my boss’s boss, and then they switched CEOs. I was wondering what was happening. I was the only person in Brazil. So they had me come to New York and have a meeting with David Karp, and the new COO, and I presented all our results. They said it was amazing and great. They said they were not doing international anymore but had all these partnerships in place. They needed to get stuff in order here in the US and didn’t have time for international.  At that point, I could try to stick with Tumblr and come to the US, but I obviously didn’t understand what was happening to the company at the time.It seemed like they were firing people left and right. They were changing plans. I did not feel that it was stable. I spent a whole year developing a bunch of relationships in Brazil, so I knew all the media players, all the tech people and people also knew me from the bunch of talks I had given. I had been in a bunch of interviews and magazines. I thought there’s a lot of companies like Tumblr that seemed to be interested in growing in Brazil and other Latin American American countries and have no idea where to start. And maybe I could just do this, maybe I could just do this growth thing for other tech companies, too. So I launched my own consulting company,

Shamanth

Which was called?

Gina

It’s called Global Ginga. I thought it was a really good idea. Ginga is a word in Portuguese that means your ability to maneuver around situations like soccer is big in Brazil. So people who have genes that can play soccer really well, or dance really well and that’s basically what I was doing. I was helping companies come to Brazil, and figure out what they had to do, no matter what, and I was executing it for them. And then Ginga was also a play on my name, which is Gina G. I started this consulting company and  Duolingo was one of my clients.

Shamanth

Great. So you started Global Ginga, and you started having Duolingo. And you eventually started working full time with them. And presumably, that’s when you came back to America. Can you set the scene and tell us more about that? 

Gina 

I came back to America for that. And also because I didn’t want to be kidnapped.

Shamanth 

You were in all the magazines?

Gina

For that reason, and also Brazil is not safe. I almost got kidnapped once but it had nothing to do with who I was. It was just bad luck. I was in a really nice place at the wrong time. 

Shamanth

So you escaped kidnapping and came to America. Tell us What Duolingo was like when you joined?

Gina

I was employee number 20. And it was very much like the startup that you’d read about. They ordered pizza at night, and they stayed late. It kind of felt like a college environment. In a really tiny office in Pittsburgh. They had 3 million users, which is honestly a lot. It’s not like they had  zero users. And this is because Luis Von Ahn, our co-founder and CEO, is this amazing guy who created the captcha and reCAPTCHA. And he sold two companies to Google and he won a MacArthur Genius Grant. And then he gave a TED talk about Duolingo and its mission, and that drove a bunch of people to sign up for the beta, waiting for it to  become available. That really helped at launch. So we had those 3 million users. They were all focused in the US. Nobody that I knew had heard about it. My parents were like, what are you doing?  I was in Brazil, so they hired me as a consultant. I had never met any of these people in real life. I talked with them through email, Skype and stuff like that. 

Shamanth 

3 million users. And I imagine the product was also relatively basic. From what I’ve read. I know that your day one retention, which is the percentage of people who came back, the day after they first started using that app was like 15%. So if 100 People installed 15%, would come back in a single day, which isn’t very great. So when you and your team looked at that number, what did you think?

Gina 

To be honest, I didn’t get the app at that level, they did a fantastic job already of optimizing. So it did start out at 15%. What I walked into was an amazing product team. And they spent tons of time AB testing. So I can’t take credit for all of that. My main role, when I came on, was basically driving awareness and building the brand and helping them launch all over the world, and get people to find out about what this was, and then find other ways to get users like government partnerships, et cetera. It was not product related. Two years ago,

they asked me to  take over growth. This was not an official thing to Duolingo, yet. They had  a product team that was doing AB testing. That’s when my team came in. We looked at all of the sub metrics that were never really paid all that much attention to. We wanted to try and optimize not just grow the day one but also the day 14, the 30. The plan was to look at referral, our activation, take a closer look at emails and notifications and take the company to the next level.

Shamanth 

I would love to dig into all of that. But I’m curious, was there a point when after you came in, you realized, okay, this thing could become big, this thing could become the mass market product with 10s of millions of active users and hundreds of millions of downloads that it has today? Is that an inflection point that you can think of?

Gina 

I joined the company full time, because I really already believed in it. Although I didn’t really know anything about products. Looking back now that I’ve actually worked with the product, I knew nothing. I just thought, “okay, this is cute.” But what really sold me on it was the mission. 

I’m not American, and I would never have gotten all the opportunities that I had, had I not learned English at a very young age. And learning a second language, like learning English in developing countries can double or triple your income potential. I felt like I was in the top tiny percentage of people in Brazil, who would ever get the chance to do that. Just because I came from a good family. I believe that by investing in that I could help change millions of people’s lives.

I believed that and then I also really believed in the team and especially in the founder and CEO, because he was obviously just fascinating. A genius, detail oriented, really attuned to everything. And every single person I talked to at Duolingo was just the best in class in whatever it is that they did. And I hadn’t felt that way about a team before.

Shamanth 

When you’re working with amazing people, you want to stay. You said you were in charge of growing the app and spreading awareness. I’d love to dig into that in a little more detail. What was your approach to growing the app after you came in? What were your first priorities?

Gina 

In terms of the growth or the product team or originally?

Shamanth 

Originally as soon as you came in, you were charged with growing awareness for the product from what are some of your first priorities at the time? What did you first do?

Gina

I first drew on the playbook that I had used for Tumblr and seemed to have worked, which was that I knew all the journalists and editors that mattered in the top verticals here in Brazil.

I thought to email them and tell them that I’m in touch with this amazing guy who created the captcha and did all this stuff. I feel like Luis at the time was kind of trying to hide it because he didn’t want to talk about the captcha anymore. He was so over it. It’s been like 10 years already. 

I knew that the nerds and tech would totally geek out on the fact that they could meet the guy who had invented that. I worked with them with a team to figure out how to position the mission, how to position what he was doing, and it wasn’t too hard because I was sold on it. I really believed it. I reached out to all these different journalists and I helped Luis. I set up a lot of speaking opportunities for Luis in Brazil.

So he came and we did a little tour with him. 

We went and took him to  this big conference where he gave the opening speech. And then I took him to this big organization where he gave an important talk. I had already reached out to all these journalists saying, this guy who’s not only amazing, has actually been invited by the most important institutions in Brazil to come especially to give a talk, even though I had kind of organized that behind the scenes. I asked them whether they would like the opportunity to speak with him? And that worked out super well. 

I only could do that, because we already had a lot that I could work with. But that’s kind of what we did. And it worked so well, in terms of the numbers that we saw in terms of new users that he then came back to do a second round of interviews, some months later. So for us, that was super impactful. And then they actually hired me full time, they said, “Okay, this seems to work for Brazil. Can you do the same thing in Chile and Argentina?” Then we went, and did the same thing in Chile in Argentina. So then they said, Okay, well, can you come on full time? And maybe we can do this in other countries? And so then the step was, if we’re going to think about other countries, where should we be thinking of? How are we going to prioritize this and then go into those countries myself, and figuring out how to do that same thing, basically get attention.To be able to talk about Duolingo, and Luis and our mission in places like Turkey and India and Japan and a bunch of other places that I’d honestly never been to in my life.

Shamanth 

Wow, that’s such an unconventional approach to growing an app or a product, simply because most apps and products when they decide to grow internationally, what they do is they go to Facebook, they go to Google, they put money behind an ad campaign, and they see growth that way. Did you consider alternative ways to grow? Maybe in addition to the immediate pressed trips that you were already doing?

Gina

Not paid. We had agreed not to do paid from the very beginning, because we were not profitable. We didn’t have revenue and venture backing. We didn’t want to start paying for users until we knew that our retention was where we wanted it to be. We knew that we had an LTV for our users, which is finally the case now. So we started doing paid acquisition actually, quite recently, but we were doing other things other than the PR.

In terms of growth, I think the PR helped a bunch and international launches, etc. Like I said, the team was optimizing the product and making sure that it was actually something that people were happy to share with each other. And once they discovered it they would tell their kid or their friend, so that made a big difference. 

We also did other unconventional things like, partner with the government and have schools use Duolingo officially. We worked with the Department of Education in New York. We got a lot of  inbound random emails and just started thinking about what we can do that would really impact a lot of people, and that can actually get people to take us seriously, because Duolingo is an app and it’s very fun and cute. So we needed to make sure that even serious language learners knew that this was a real way to learn language. We did a lot of that and I think that it made a difference.

Shamanth 

So let’s say you went to Turkey or India, you do this press tour. Luis speaks, you speak to a lot of media. And let’s say you get out of the country, you go to your next country. Would you notice that the user growth in the country that you just left would start to slow down?

Gina `

Absolutely. But because retention was in a good place, when we started, what we saw would be a big bump, we’d get featured on the App Store and all this press and then we’d see this huge bump. Then it falls down, in terms of new users per day to about 40 to 50% of that bump, which is still way higher than it was before. In terms of our users, we were retaining them and they were telling other people and that engine was working. That’s why we kept using that same technique across countries. Now, there are some places where it was more effective than others. And there’s some markets that were just really hard from so many different perspectives.  I was working without agencies, I was just emailing the media by myself. And I did that, because I just haven’t had the best experience working with agencies in terms of results. We went to Korea, and suddenly, nobody speaks English, and they don’t really understand what you’re pitching, and maybe you’re coming at it from the wrong perspective. So there, I really do feel like there are places where I didn’t hit it out of the park. But then there are places where we did the same thing like in Turkey, and suddenly, our numbers skyrocketed.

Shamanth 

Can you think of any moments or instances that you’ve found particularly challenging during this phase? While you are touring all these countries?

Gina

Oh, for sure. There are so many. From a very technical perspective, getting journalists is like a big deal. They shouldn’t just say “It’s a top app you should use”, but they should really be interviewing and writing about it. That takes a lot of work. I was really particular about making sure they included a link to Duolingo. Because we would see a very big difference in traffic if people included a link in the article versus if they didn’t. So it has to be really nice and kind of big. 

In terms of really challenging, I think it was when I went to China to launch Duolingo. I was there for a month by myself and that was trying. I speak some Chinese, but I’m not by any means fluent. Figuring out how to get by it and figuring out connections there, it was such a foreign world, and you needed to have ties with the government.

When we launched, we got a million downloads the first day, which is huge. Then our app got blocked by the government. There was nothing I could do about it.  I had been in touch with Apple, and they were like, What is wrong with your app? And I was like, it’s not on our end, but they wouldn’t really believe us. And then  all the people who had downloaded the app started rating it like one star because the app didn’t work. So it was just so frustrating, after all that work to see that happen and not be able to do anything about it.

Shamanth 

Yeah. And you were alone by yourself in China. And I imagine that can be isolating.

Gina 

What is most isolating is not only the fact that you’re in a place where nobody really looks like you but also it’s a time zone where nobody was ever online when I was awake.

Shamanth

To switch gears a little bit. I’ve admired that you guys have had such an unconventional growth strategy. As you said, you’ve done PR, you’ve done extensive media outreach, you guys have done extensive experimentation. I do know that you guys have been mentioned quite a bit by Tim Ferriss, who’s a peer of mine as well. 

To the extent that you’re comfortable sharing. Can you speak to how he’s impacted Duolingo?

Gina 

He might have had more influence before my time, directly with our co founders. So I can’t speak to that. He was an early investor. So that, of course, helps. He mentioned us several times and he does have a very serious following. But the most impactful thing that happened was, I had a chance to have breakfast with Tim Ferriss at his house. And breakfast with Tim Ferriss was really just tea because he was fasting. Which was very disappointing because I thought I was going to eat. That was a really cool conversation that we had about  the branding and the positioning and what we should do. 

During the conversation I said, “Hey, Tim, maybe you could just interview Luis for your podcast. “And he was like, Yeah, of course, obviously. And I honestly did not understand the impact that that interview would have. Tim Ferriss was a big deal. He’s a Silicon Valley big deal. He’s awesome and it was good for us. 

We went to India  six months later and people were like, “Oh my God, you’re the guy from the Tim Ferriss podcast.” That was really big for us and super worthwhile. And honestly Tim Ferriss in my humble opinion is one of the best interviewers that I’ve listened to. He is just so good at getting people to talk relevantly and carry a conversation. The interview he did with Luis was really rich and different.

Shamanth 

Absolutely. I loved his interview with Luis. And he is very much an inspiration to me. 

So at this point, you’ve been focused on media outreach, you’ve been focused on getting the word out about Duolingo. How did you transition to growth and experimentation?

Gina

They were basically throwing me whatever wasn’t product, because I was one of the very few people in Duolingo who was not an engineer or designer. If an opportunity or a business development assignment came up, they would ask me to check it out.  I was also running social media, a lot of our copywriting, anything that was not around product. It was kind of all over the place, then it became a lot more of what I described, along with social media and communications for the brand, and branding. And then finally, two years ago, Luis and several of the co-founders came to me and said, “Look, we had a conversation with the board and they feel like someone needs to own our growth numbers.” And I was like “Great, what’s a growth number?” And they said,  “So far, Luis has been the one who really tracks our DAUs, or MAUs, user, retention numbers, etc, and makes sure that things are in order, we need someone to do that. And we discussed whether we should hire or promote internally, and we think that you should do it.” 

And I thought, that’s great. I have no idea what that means. And then went back and Googled everything I could possibly read about it. I really just went into super study mode. I was living in San Francisco at the time. So I also reached out to a bunch of people who were really knowledgeable about growth. The guys from CapitalG really helped a lot because they made intros.  They’re one of our investors and I got to pick their brains. 

Originally it was a very tiny team, we had an engineer, a designer and that’s it. All i had to do was look at some numbers and try some stuff out. 

I remember the day our engineer decided to add a badge to our Android app, a little badge that says that there is something new on the app, and you have to click little red dots to find out what it is. He wanted to do that. I thought it seemed kind of spammy but I told him to try it. He just went back to his desk and did it anyway.It was like 10 minutes of running code, seven lines of code. And that was one of our most impactful experiments ever.  It was really a lot of experimentation, and reading and trying things out and thinking about, like, what are the most impactful places that we can touch? And what are the highest ROI things that we can work on?

The team really grew over time, and ended up being about 12-13 people, including  product manager, designers, engineers, and the data analyst.

Shamanth 

What I love about that story is that you started from basically scratch without any background and what you might call hard quantitative skills.

Gina 

I will say that I am proud of that. However, a big component of that is just that I really got to work with excellent engineers.  People who really were so smart.  I definitely know, I brought value to the table, but I couldn’t have done any of this without them.

Shamanth 

When you started running these experiments, how did you pick some of these experiments? What were some of the experiments that you first tried?

Gina 

We had a dump of experiment ideas that had come up from different people across the years that we never had time to try. I went through like three different Excel documents that one person put together and an email someone sent someone else. I tried to gather all of it, and then we sat down and looked at all of those and  prioritized them in terms of  how much effort it’ll take to run each experiment and what we think the potential gains could be. I need to credit Avinash Verma, who was a senior product manager at Duolingo at the time who really showed us how to do that. That’s how we started thinking about it. 

We started going through the list. Once we started getting the hang of it, and I had been reading a lot of articles, we started coming up with our own ideas. We just added to the list and continued to prioritize. That’s how we’ve done it for the past two years, we have this ongoing list, prioritizing in every quarter. We look at our top priorities and then think of a theme or something to focus on. For example, we’re going to improve the team retention this quarter, or we’re going to  improve the social experience. We would then find experiments that we think are most impactful. That helped us hit that metric, we would set OKRs and then we would go for it. Depending on the results of each experiment, we could then iterate on those ideas or just scratch them and move on.

Shamanth

What were some of the experiments that failed?

Gina 

I would say 50% of our experiments failed.

I remember when we decided to redesign the newsletter. Anything that involves design was so much work.  I think on one hand, it’s because I couldn’t do it myself. I can write copy, but I can’t design. I need to rely on other people. The design standards are extremely high. In order to get something approved, it takes a very long time. We did all this work, and then it had zero impact on anything.

I think that our biggest failures weren’t these disappointing experiments.

Our biggest failures were when we ran experiments, and they either worked or didn’t work. And then later we realized that actually, we weren’t tracking metrics properly. That, to me, is the most frustrating thing for the whole team. For example, for six months  we were working on sign-up/ sign-in things in terms of  optimizations, and then we realized that when someone who already had an account, but forgot that they had an account created a new account, the system wasn’t linking accounts. So they were creating new accounts that weren’t really new accounts, and this was inflating our number of new users and potential. We realized that we didn’t know how to fix this. We also did not know how many experiments this affected. 

Shamanth 

That can be frustrating when you realize you don’t know if everything you’re reading is right or not.

And what were some of the experiments that you were proud of.

Gina 

I can tell you about the experiments that I think were most impactful in Duolingo and also that I’m proud of. I’m proud from the perspective that we did a good job again, none of this was my work alone. It all comes from brainstorming with the team and teamwork. I think one of our most proud things in the growth team was creating the badge system on Duolingo and  adding badges so that as you go you earn badges. The early riser, late bird and the the night owl. Also if you buy something, you get something and coming up with little skewed names. Coming up with how the badges and how the system was going to work. I’m really excited about this because it could have been a humongous flop. It took us  four months to do this. Afterwards, we have to figure out where on the app they’re gonna live? Are people going to be able to showcase them? Are other people gonna be able to see them? What is the mechanism? Are they going to be like, teared or not. They came up with the design of the coffee and it really took a long time and implemented the whole thing. But it ended up being extremely impactful for us, not in more ways than we really imagined. Not only were people coming back more to Duolingo because they were getting badges. They were doing stuff that we were giving them badges. That wasn’t  just benefiting our team but benefiting other teams at Duolingo like revenue. You were like, buy something in the store and get a badge, add a friend and get a badge. We were helping the social team.  They wanted us to help with their metrics too. And so then we iterate it and actually spend time doing a tiered system so that we could have a better incentivized format, and really nailed the ones that were bringing in the most results.

Shamanth

And you’ll realize how much people value even digital goods like a badge. It doesn’t exist. It’s fully virtual. But it’s so important. It’s so cherished and valued by a lot of people.

Gina  

Yeah, it makes people feel good and a lot about gamification is that it’s just like giving people that sentiment of accomplishment.”I feel good. I don’t know why, but  I’m just going to keep doing this.

Shamanth

 Absolutely. You’ve spoken about how Duolingo has used gamification quite extensively. How did this begin?

Gina

Duolingo was originally launched with the intent of being a game. It was baked into the product from the very start before I was at Duolingo. The idea behind that is that learning a language is very hard, because it takes a very long time. It’s not the kind of skill that you can dedicate a weekend to and then be done. I studied Chinese for three years, and I still barely get by. So you really need to dedicate years of your life to this. Also binge studying one day a week doesn’t work, you need to do it very regularly.  Add that with the fact that now we’re telling people, Hey, you can do this by yourself, no teachers are gonna be making you do it. Your parents are not gonna be angry. If you don’t do it. You’re not paying money, so you’re not gonna feel guilty. And it also competes with your Facebook time.

Shamanth  

That’s a very high bar. Go ahead.

Gina 

That was a big problem that the team wanted to solve from the get go. If we’re going to try to get people to learn a language, what are we going to do to actually get them to do it? Because the thing is to sign up. A lot of these MOOCs have exactly that problem. They get a lot of interest in signups.

People are like, yes, I want to take Harvard classes, then the retention is really low. Because it’s so hard to convince yourself to keep coming back. And to complete a course that you don’t lose anything if you don’t complete. The thought process was well, but people do spend all their extra time playing games like Candy Crush at the time, Clash Royale when they’re in line, when they’re waiting for a meeting with their lunch. That’s what they’re doing with their time. What can we do to make that be Duolingo?  From the very beginning, it was designed to be like a game, but it really took all these years. And I think it’s still a process to improve, because gamification isn’t just something that you throw onto a product. It’s adding layers and layers of things that make it more fun, like the points system and how you can  pass your friends, and you can unlock things. Now there’s a mascot and you can earn points and you can earn digital currency. Now you can buy things with digital currency, and you can get badges.

Shamanth 

It sounds like you guys were very intentional about making sure people had positive reinforcement at every step of their progress in learning a language and therefore made language learning so much more effective for them.

Gina

And one thing, I will say that our team did do a lot, where we started having focused brainstorming sessions, which were like, “You play this game you play that,” we would just look at the top games in the app store in a particular time, and divide among the team, then we’d have people in our meeting be like, I thought it was interesting that they did this. I like this sign up, and I liked this way of introducing the game. And then we would think about which of these do we really want to pay attention to, we think could be most impactful, and how could we imagine that overlaid on Duolingo?

Shamanth 

Is there an example that you can think of? An idea that you guys borrowed from a game?

Gina

I would say that probably all of our ideas. Even from the start, for example the little hearts that became a strength bar, but you lose hearts as you go. The badges – we didn’t come up with badges. That’s a Foursquare thing. But that has been used by a zillion other apps now.  

Shamanth

Great. It sounds like you were very disciplined about studying what was out there and borrowing very liberally and also adapting all of these to Duolingo.

Gina

Yeah, I think that there’s no reason to reinvent the wheel. 

We weren’t stealing. We were taking ideas and overlaying them on our own context.

Shamanth 

Was there an example of an idea that you saw in a game that you realize wouldn’t be a good idea for Duolingo?

Gina 

There’s a lot of stuff that we would discuss. And someone would say, I love how this is like that. And I would think, I really wonder what the metrics are saying, because there are some things that you just really like, and feel good or look good, but then you look at the numbers, and they’re actually completely neutral or negative. And there are things that we saw in other apps that we had already tried, and we knew wasn’t weren’t positive for us. One thing that we struggled and sort of continue to struggle with is that one of the main things about Duolingo is that it’s extremely simple. And that means design is simple. 

The user experience, the flow. Everything is supposed to be intuitive and clean. And we’re very thoughtful about not adding clutter and things. We started seeing that games were adding 70 different layers of point systems. There was just so much going on at the same time that we often struggled with. Are we underestimating our users? Should we be throwing more things at them at the same time or not? Should we experiment? Does this completely veer off our brand? Is this going to maybe improve our metrics, but make our user experience and design as good, which is something we really value? And so that was definitely an ongoing discussion.

Shamanth 

You’re right, because most games have multiple currencies, multiple meta games there’s a lot of direct gameplay, and there’s a lot of complexity in most smartphone games.

How did you guys decide that? Okay, that’s not a path we’re gonna go down on?

Gina 

We spent two years really optimizing stuff. And originally there was a lot of low hanging fruit. We would change the notification, copy or change the email subject line, or change the timing of the notification so that we would see 5% bumps, which is a lot in our numbers. And we did that for two years. The badges was much more of a long term investment. And then as we started looking at these very complex aspects to games, and having this discussion with the rest of the company of like, should we even be trying this? Does it go against our design principles, etc. 

We started realizing that at this point, we weren’t a growth hack team, in the sense that we, we weren’t looking at hacks anymore. We were looking at changing products, fundamentally, we were looking at adding huge features, like the badges were a huge feature. But that was kind of a one off. Yeah. And it didn’t make sense for that to be done by a team that’s just focused on growth. But a team that was more focused on  the experience, the gamification, the social aspect of Duolingo, so that it wasn’t just like, optimizing metrics for the sake of optimizing metrics. And we didn’t just think that it was high ROI, because all of these ideas would take a significant amount of rain to build. So we ended up reprioritizing.

Shamanth 

Just to switch gears a bit, you guys do a lot internationally? Did you guys have to adapt the product experience itself to some of the international markets that you guys were in, for instance, if you’re in Saudi Arabia, I would imagine this person’s language is different. But also this person’s expectation of the user experience is going to be very, very different. How did you guys adapt the product and the user experience to hundreds of different cultures around the world? 

Gina 

Our answer, I think, is very controversial here. Yes, I agree. People have very different expectations of what an app should look like & feel like. We chose to ignore almost all of that. And so what we did is we localized, obviously, so launched the app in a new country, we would basically just make it available in the language of that country. And then we would  launch an English course, for example, for speakers of that language.

And a lot of our courses were created by volunteers, actually using the Duolingo incubator, which is like this platform we created for people who really wanted to create a course for that language, either because they wanted to help people in their country get ahead or because they wanted to preserve their language. And they would create content that was different anyway, so there was a little bit of natural localization in that aspect beyond just the translations. 

But then when it comes to the product, we knew that  it might be a good idea to start really trying to make it specific to each market, but we also knew that it would add so much product complexity. Because now, everything you do, you need to think about, how the app looks in all these different countries and all these different platforms and modify and do different designs,  and different flows. So we just didn’t. And the truth is, and this might be controversial, but in my experience working with Duolingo, I think that people are more similar than they think they are. In the world. I think that when you go to a country, and someone says, in this country, like in China, or in Brazil, people really like free things. I get that there’s something interesting here. There’s a cultural nuance, and we could explore. But I don’t think that we need to completely reinvent Duolingo in order to do that. 

So that worked for us in almost every market that we launched, with exceptions. So I would say the two major exceptions are China and India. Those are extremely difficult markets. I’m sure that that is not news to anybody. So for India, that’s the first place where we really and that, again, full disclosure was not my team. I was, you know, I worked with them. But our Director of Engineering led that effort along with the product manager. And we basically went to India, and they conducted a bunch of interviews, and I talked to reporters. And we learned as much as we possibly could about internet usage in India and how people were using the app and found that there were actually fundamental problems that prevented people from using Duolingo. If we didn’t do anything to the product. Like, for example, in India, a lot of the phones come pre-programmed in English. 

So when you buy your phone, it’s in English. And it’s just how it is. And also, typing in Hindi is supposedly very difficult. So a lot of people get these phones, and they’re in English, they don’t actually speak English. They just kind of learn how to use a phone. And then what happens is Duolingo notices what your UI languages and it adapts to that. So people were now seeing Duolingo in English, and they didn’t see the option to learn English, because from English, we don’t teach English, we would teach French or German. Our English from Hindi was never seen by anyone in India, we had to change the flow there specifically to match that.

Shamanth 

So you guys had a default flow in the local languages in India.

Gina

We actually launched an English from Hindi course. And we’re working on other languages now. And so what we did is , if you’re in India, and your phone is set to English, we show “learn English” on Duolingo. 

Shamanth

That makes sense. I think there’s just been a huge influx of smartphones in India, and I think there’s a lot of people that don’t necessarily have English as a first language. But they absolutely want to be learning English. And I think that’s a huge opportunity for you guys.

Gina 

We also learned that a lot of people buy their phones from little shops, and they don’t actually download their own apps, the shop downloads the apps. I was talking to this very intelligent, well read, amazing tech editor and he was showing me his phone, and I told him to download Duolingo, so that I could show him something, and he did not know what I was talking about the app store.  Just a very different way of using technology. If you want to grow in a market like that, where it totally makes sense for us, not only in terms of number of people, but also our mission, there’s a lot of people who could benefit from learning English in India, like significantly could like really transform their lives. And then you do have to start paying attention to things like that.

Shamanth

Absolutely. And you guys are making such a huge difference to the world. And as you mentioned earlier, learning English in a country like Brazil or India can make a huge difference in the kind of money somebody makes, and can pull people out of poverty. Gina, I know we’re brushing up against the end of our time together. This has been wonderful. I would love to start to wrap up here.  Before we go, can you tell people how they can find out more about you?

Gina 

Find me on LinkedIn. My username is Gina from Brazil.

Shamanth  

Wonderful. I will link to your LinkedIn in the show notes. But as always, this has been very, very exciting to speak with you, Gina. Thanks so much for being on how things grow.

Gina 

My pleasure. Thank you so much for the invitation.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify, Google Podcasts or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!