📲 How Slack reversed a declining mobile growth trajectory on mobile by setting up cross-channel attribution – with Holly Chen, former Global Head of Digital Marketing at Slack. 💬

Our guest today is Holly Chen, growth advisor and the former Global Head of Digital Marketing at Slack. Before that, she led growth for Google Store & Google B2B websites, served as a product manager at Gucci, and had a brief stint at the UN.

I’m excited for this interview because we look at one of the most admired brands of the last few years. We take a closer look at how they looked at mobile marketing while accounting for the numerous complexities of B2B marketing – including longer sales cycles, multiple touchpoints before sale. In the process of orienting their growth to mobile, they turned around a declining mobile growth trajectory – in what I find a very impressive effort.

KEY HIGHLIGHTS

🧗 How mobile fits into a B2B buyer’s journey for Slack – and why it’s challenging to measure.

📉 Why a decline in mobile growth in 2017 was puzzling for Slack – and how they realized this was an opportunity.

🚶 The first steps Slack took toward capitalizing on the mobile opportunity – and the success metrics they picked.

🔗 The interdependence of mobile and desktop products at Slack.

⏱️ How Slack attributed and measured the impact of the mobile marketing program.

🖐️ What multi-touch attribution is – and how Google Analytics can be a starting point for products that are considering adopting multi-touch attribution.

🏢 The infrastructure that needed to be in place for multi-touch attribution.

🤔 How Slack’s teams thought about mitigating the risks of multi-touch attribution.

👨‍💼 How Holly and her team approached getting executive support for the mobile marketing program.

🗓️ Some of the day to day tactical elements that had to change as Slack oriented itself more toward mobile.

🔥 The impact of the shift toward mobile.

KEY QUOTES

No glamour for mobile in B2B

So mobile typically, almost is like an adopted child in a lot of B2B products. At Slack, we saw this from both macro perspective and the local perspective.

Challenges faced with mobile

We also recognise there’s a lot of challenges in mobile, especially for B2B because typically the traffic is lower volume, in comparison to desktop. Typically, for B2B we see about only 10 to 20% of overall traffic was on mobile platforms, and even lower percentage for conversions. And typically the quality of mobile traffic is also lower than desktop because most of the conversions will happen on desktop. And it’s also harder to measure – with the mobile touch point versus a desktop touch point, it oftentimes is hard to connect the dots.

There is difference in desktop and mobile growth rates

At Slack, around 2017, we saw that the new team creation from mobile declined, but at the same time, our desktop team creation experienced a really high growth rate. So we thought, well, this is not necessarily normal, we were expecting, if the market demand was increasing, both channels should be increasing, instead of one increase and one declining.

Start with the unknowns

So we first laid down all known questions we wanted to answer and secondly, define success metrics. What do we want to achieve from this programme? So what we determined was we do want to drive top line growth with mobile acquisition channels, but with a realistic CPA for this effort.

Mobile multi-touch is complex

One of the key challenges for mobile is the difficulty to connect all the touch points across devices until the conversion and the activation. So, there are two things. One is multi touch attribution for cross device which I can talk more about, and two is actually for the programme, our goal also, we revisited every three months and six months and to see the impact of it.

Transitioning gently into multi-touch attribution

The easiest way to start looking at this is to look at your Google Analytics report, there is the path to conversion reports and there’s multi funnel report. Google also has the attribution model comparison tool, which we found really helpful to initially validate some of our assumptions.

How combine multiple sources of data

What we found is that the walled garden is becoming smaller and smaller and tighter, it’s harder and harder to use a black box attribution model outside of our internal data warehouse. So what we did was to use our app server to capture the raw data points. So for desktop, we used the DoubleClick suite, DCM (DoubleClick Campaign Manager) as an ad server. And then for mobile touch points, we use our MMP and both of them send raw data points to our internal data warehouse. And then using email addresses, we connect the dots between the converted users in order to apply the MTA (Multi-touch attribution) model on top of it.

Recognizing that multi-touch attribution isn’t perfect

So multi touch attribution is not perfect. It is one of the approaches to measure the effectiveness of marketing and sometimes, MTA can be arbitrary. So I would recommend combining MTA with incrementality studies and really understand the incremental and true value of your touchpoints. Ideally, you can run incrementality studies by channel, so you know the incremental value by channel and new methodologies like ghost ads that came out in the recent years.

The importance of agility

Revisit your model and how it works every, I would recommend like 6 months or 12 months, because your user composition and behaviour may change. Especially for a fast growing company like Slack, our users preferences or their behaviour could be changing very fast every 6 months or 12 months. So, as marketers, we need to be nimble to adjust our approaches.

Tactical changes when mobile is secondary

We shifted a lot of our efforts from purely top of funnel top line growth to really understand the cross device impact. So for example, if someone downloaded a desktop and didn’t have a mobile mobile app installed, traditionally we only had email or in app notifications, and we started to experiment with the retargeting efforts to these new existing users. So over time, we did adjust our approach.

FULL TRANSCRIPT BELOW:

Shamanth: I’m very excited to welcome Holly Chen to the Mobile User Acquisition Show. Holly, very excited to have you.

Holly Chen: Very excited to be on the show. Thank you, Shamanth.

Shamanth: Yeah, I think we’ve only been connected for the last couple of weeks. But the couple of times that we’ve spoken, I’ve very much enjoyed both the humour that you’ve brought to everything you do – humour and happiness you bring to everything you do, and also just the sheer depth and insight you bring to a lot of the work you’ve done. And for all of those reasons, I’m excited to dive into our interview today.

Holly Chen: Yeah, I know your humour type as well. So yeah, humourous like humourous. That’s why we get along.

Shamanth: Indeed, yeah, yeah. Cool. Holly, we’re gonna talk about a lot of your work at Slack. And specifically, how you guys worked with mobile, how you guys really oriented a lot of your marketing towards mobile. Just to take a 10,000 foot view, how does mobile fit into a buyer’s journey at Slack? How did you and your team identify this?

Holly Chen: Yeah, great question.

So mobile typically, almost is like an adopted child in a lot of B2B products. At Slack, we saw this from both macro perspective and the local perspective.

From the macro level, we understood especially in recent years, mobile has been more and more important for a B2B user journey and purchase decision process. Especially there were new reports and research that came out, says about 70% of the buyers under 40 years old and 50% of buyers over 40 years old use mobile before buying a B2B product. And that’s more than, you know, half of users use mobile to do their research. And then in terms of B2B queries, so people who look for B2B type of products, 50% now are made on mobile and there’s an estimate to say like about 70-80% will be made this year or next year. So mobile is increasingly more and more important in the B2B purchase funnel, but we know very little about it. And we observe that mobile can accelerate the time to purchase. There was research saying that it can accelerate up to 20 to 30%. So we know intuitively mobile is very important, people are on their mobile all the time, and especially for decision makers who use mobile a lot as they commute, pre COVID that is.

But like at the same time,

So that’s the macro level, we understand there’s opportunities, but also risks.

And then on the local level

At Slack, around 2017, we saw that the new team creation from mobile declined, but at the same time, our desktop team creation experienced really high growth rate. So we thought, well, this is not necessarily normal, we were expecting, if the market demand was increasing, both channels should be increasing, instead of one increase and one declining.

We also knew that the mobile teams showed lower lifetime value. And we understood that the mobile product was also resource constrained. So there’s a lot of reasons why we didn’t do mobile earlier. But at the same time, we were also doing a lot of research about lifetime value, on understanding the user journey better. So what we found was for a team creator that created a team on desktop, if they have a mobile usage, their lifetime value is 5 to 10x more than just having one touchpoint, one platform within their lifecycle. So we knew that this is very important, and potentially what we want to explore not only mobile acquisition, but also thinking through mobile’s role in the lifetime of the user journey.

Shamanth: Right. So clearly, you guys saw that, at a macro level, B2B buyers are using mobile, which makes sense because broadly, everyone’s using mobile. And I would also say Slack when I think about it. It doesn’t come to mind as a B2B product, but it absolutely is one. So you are like – right, in the broader world, more and more B2B people are using mobile. And with Slack specifically, you’re like, hey, mobile isn’t growing. And why is it? It’s not because people aren’t using mobile. And if anything, mobile users are more valuable. So you’re like, look, where is the disconnect?

Sounds like that’s how you guys were thinking about it. And once you guys notice this disconnect in terms of how valuable mobile users were, and how much it was growing, which reflects, I imagine reflected how much you were investing in marketing mobile, and promoting mobile to your users, what were some of the first steps you guys took when you said, right, I think we think mobile could benefit from more marketing investment/ effort. What are the first things you guys did?

Holly Chen: Absolutely. I would say the first thing is to have a realistic and holistic perspective around the role of mobile and this exploration. Because to be honest, we still don’t know the value of mobile, for B2B in general for Slack, especially, because as you said, it’s a B2B product, it’s also prosumer. So the user behaviour could be different from a normal like a consumer consumer. And it could also be different from a traditional enterprise buyer profile. So we didn’t know exactly what would happen. And two is the cross channel impact we still need to understand better, but we didn’t know that – for example, from a mobile search perspective, we were doing a lot of desktop, you know, brand search or like non brand search.

But on mobile, we weren’t doing anything. So like a lot of competitors are also bidding on Slack keywords. I would say okay, like, at least from the brand defence perspective, we need to start doing something. And more broadly, how can we use mobile more effectively as a channel? So we first listed all the unknown questions that we don’t have answers for. So for example, should we drive the traffic, mobile traffic to a mobile web landing page, or directly to the App Store, right? Like for a gaming or e commerce app, it’s very like typically, it’s better to drive to the App Store, but for B2B products, is it better to lead to a landing page where people can learn your product more and educate them more and then they would have a higher conversion rate…. like these are all assumptions we have no idea that whether they would work or you know things like, oh, how would organic search work vs paid search? Or like which MMP should we pick?

So determining the goal for the programme is the second and very critical for us to align the team’s effort and I would say step three is to get buy in and cross functional support. Very important to get executive buy-in – and setting expectations, and communicate what we want to achieve. But also, mobile is something that cannot be done in silos. Mobile engineering, data engineering, growth product, growth engineering, analytics, IT, finance, privacy. So all of those teams have played very critical roles in making this programme successful. So really getting everyone on the same page and understanding the value and the purpose of the programme.

Shamanth: Yeah. Even though you guys looked at mobile as a key new initiative, mobile wasn’t siloed as you said. Because the mobile product is intertwined with the desktop product. It is the same product, which is wholly unlike a lot of mobile app companies. If you’re a game, you don’t have to worry about the dependency on desktop. So I imagine there were a lot of interdependencies that you guys had to be very cognizant of.

Holly Chen: Absolutely, yeah. You know, just from a user experience perspective, how our messaging is working for mobile versus desktop? Is it the same? Are we emphasising the value of having multiple devices? And you can check things at different places and on the go? So starting from messaging to creatives to targeting, to measuring, all of those are very cross functional.

Shamanth: Yeah. Right. And you did say that one of the key components was to understand what success metrics are. And you’re like, look, that has to be top line growth. And yet, I imagine one of the key challenges was to attribute top line growth to mobile per-se. Can you speak to some of the challenges involved in essentially attributing top line growth or outcomes to the effort and investment you guys were making it over?

Holly Chen: Yeah, for sure.

So initially, our goal was to drive top line volume. So like a pure number of top line conversions from mobile but then, as the programme went on, we looked into the conversion rates for quality conversions. And we define quality conversions as potentially teams that are either more likely to activate or more likely to engage over time to upgrade to paid. And then the goal of the mobile also went into driving more activation, monetization and optimise for cross device impact. So the programme itself didn’t stay stagnant. And the goal also changed over time based on the business goals.

Shamanth: Right. And, you know, you said one of the solutions was just multi touch attribution, just figuring out how users interact with different touch points prior to conversion. Can you tell us a little bit more about what multi touch attribution is… for people who are not super familiar with it. And at a high level, how does it work? What happens and how is it different?

Holly Chen: Yeah, multi touch attribution typically refers to understanding the different touchpoints and their role in a conversion funnel and properly giving credits to each of these different touch points. And typically it is discussed in the context of last touch attribution, which gives 100% of the credit to the last touch points before a conversion. So if someone first looked at a display banner and then did an organic search and they converted on organic search, that touch point will get 100% of the credits if you’re using last touch versus in multi touch, you are also taking into account of that impressions from that first display banner.

Shamanth: Sure. And as you were saying earlier, you did notice there were a lot of mobile touch points that were perhaps enabling the final conversion that weren’t reflecting in the earlier last touch model. So once you decide, okay, you know, we think we need to have a more multi touch approach. What was the high level approach to rolling this out? Was it like, right today we are last touch, tomorrow we’re going to just go have weights assigned to each touch point, was it overnight? Can you tell us more about what that transition phase was like?

Holly Chen: Yeah, yeah. Multi-touch is really important, especially for a B2B product. Because the conversion window is longer, people do need more information and more education over time before they can convert, rather than something like an impulse buy or a smaller ticket item like ecommerce or mobile, where a lot of the purchase happens much more closer to the last touch. B2B products typically require users to look at your websites or watch videos or potentially even go to events before they can make a big purchase item. For Slack in particular, that window is shorter comparing to a traditional big enterprise product because we are bottoms up and have a freemium model, but we still observe a large notable percentage of users convert outside of the 30 day window and on average, their multiple touch points and actually according to some of our research, 10+ touch points before someone converts. So, we understand that most touch points on both mobile and on desktop are important in that journey.

So, looking at this, like existing data points and historical data, and that confirmed our hypothesis of the direction we wanted to go. And then we looked into a lot of the research and then started implementing.

Shamanth: Right and so it sounds like if a company is trying to figure out, “hey, is this right for us?” perhaps that’s where they should be going Google Analytics to see do they even have multiple touchpoints and if this is something that they should even be thinking about or not.

Holly Chen: Yeah, like looking at your G.A. report and just seeing how your existing users behave, and you probably have an internal data warehouse that on the log level, see the user journey of your own converting users. So that’s another level of analysis you can do on your own internal data to look at, well, if we apply to the multi touch attribution model, what’s the impact? And what percentage of our users actually were exposed to other touch points before conversion.

Shamanth: Yeah. Cool. Yep. And you know, with mobile, you’re getting a lot of the acquisition data in your MMP. And with a desktop, I imagine a lot of it comes with your analytics, or perhaps Google Analytics or your internal analytics. To the extent you’re comfortable sharing, can you speak to what sort of infrastructure had to be in place to capture these disparate sources of data?

Holly Chen: Yeah. So first of all, in order to do multi touch, you need to have raw touchpoints, raw data about all the touch points that you have. And we’ve explored various different ways and we actually explored some of the out of the box attribution model vendors. But

What we found is that the walled garden is becoming smaller and smaller and tighter, it’s harder and harder to use a black box attribution model outside of our internal data warehouse. So what we did was to use our app server to capture the raw data points. So for desktop, we used the DoubleClick suite, DCM (DoubleClick Campaign Manager) as an ad server. And then for mobile touchpoints, we use our MMP and both of them send raw data points to our internal data warehouse. And then using email addresses, we connect the dots between the converted users in order to apply the MTA (Multi-touch attribution) model on top of it.

Shamanth: So, the email addresses formed the unique identifier you needed to be able to know who this user is, which I imagine is the most important part, because if you’re not clear about whether this is a unique user, then a lot of the marketing could go awry.

Holly Chen: Yeah, I would say different companies may have different unique identifiers – for us, email is part of the registration process, so it’s the easiest way for us to connect the dots.

Shamanth: Certainly. And, even as you describe it, I recognise it seems so incredibly complex, just thinking about a lot of how that multi data attribution can work. So, I would imagine that it comes with some amount of risk, because if you’re saying touch 1, touch 2, touch 3 is 33, 33 & 33%. Or you could make it 50, 25, 25%, those can lead to very different budgeting decisions. So, how did you think about mitigating the risks of incorrect decisioning around multi-touch attribution?

Holly Chen: Really good question.

So that you can set up to understand some of the more mid funnel and upper funnel channels and their impact.

So, combining MTA and incrementality studies and incorporate the findings of incrementality studies into the MTA model is one of the core methodologies that we need to do… and two is compare your pre-post and continue to monitor via GA and other tools to make sure you’re not like too far off with your efforts and thirdly is to

Revisit your model and how it works every, I would recommend like 6 months or 12 months, because your user composition and behaviour may change. Especially for a fast growing company like Slack, our users preferences or their behaviour could be changing very fast every 6 months or 12 months. So, as marketers, we need to be nimble to adjust our approaches.

Shamanth: Yeah, yeah, I think it’s a useful reminder for marketers to keep revisiting the models and back test them. I am reminded of one of the episodes of this podcast where we were talking about the daily deal space back in 2011-2012. This is Adam Lovallo who also hosts MAU, so he used to work at LivingSocial, which was a daily deals company and he told me on the podcast basically that the first time they saw that the daily deals space was crashing, Groupon, LivingSocial going down was when he was looking at a lot of LTV models and he’s like, oh, we projected these users were making $100, but they were making only $20. And I think that was the first sign. And I think it’s a useful reminder for marketers to revisit their assumptions to check what they are projecting to see if it is what it is. And like you said, with Slack, certainly it wasn’t a question that the user quality was declining. That probably still isn’t the case, but certainly user quality changing, user behaviours are changing and I think it certainly makes sense to revisit the model and evaluate whether your assumptions are right or not.

Holly Chen: Yeah. It’s like, sometimes it’s even beyond quality because, how do you measure quality, right? How do you define that? It’s sometimes if you expand to a new vertical, right, like if people who are in let’s say, a retail vertical probably behave very different from people let’s say a news agency and how they think about the use case and the value the tool delivers is very different.

Shamanth: Yeah. And in most B2B marketing, those nuances are captured, but in Slack’s case, because it’s almost as you said, it’s prosumer, I think that’s the phrase you used, I imagine it’s important to be very cognizant of some of those distinctions of how very different types of industries, different types of users behave. And also to drill into another aspect of your last answer, you spoke of incrementality. Again, to the extent you’re comfortable sharing… let’s assume you have 15 channels running or 10 channels or 5, do you like say, okay, let’s turn off Facebook ads for the next few weeks. See what happens, at a high level, is that how it works?

Holly Chen: Yeah, so typically, we like a DMA level incrementality test, just because like a pre-post, sometimes is impacted by seasonality or other factors. So it may not be as accurate. So typically we let’s say, half of the DMA’s have ads and the other half doesn’t have ads and we compare the results.

Shamanth: Yeah, that makes sense. And I think you have a fairly clear segregation of your control and test group. So that definitely makes sense. Cool. Also, something you also spoke about earlier on in this conversation, Holly, was about how important executive support was for this project, for the mobile project. Can you speak to what the process was of getting that executive support because certainly for very many companies with large teams that are embarking on a big project, so much hinges on executive support, even when the business case is fairly clear. So what would you say were some of the key factors that you would attribute toward on getting that executive support and making sure you had it while you work on this project?

Holly Chen: Absolutely. I think one of the biggest learnings is to start small, and set a clear roadmap. But remember, it always takes longer and to first set realistic expectations on this is something that we want to test and learn. And it’s not something, it’s not a silver bullet that’s going to change our landscape overnight. And two, is to think about what to focus first. So instead of expanding to all the different channels, I think about what are the quick wins that we can show or what are the early signals we can capture to understand if we are in the right direction and communicate that to the cross functional teams and also to the executive teams. So very much bring everyone along the journey of discovery. I would say continuing to optimise and adjust our approach also helped because we’re in this together and having our executive teams and cross functional support is critical for our success.

Shamanth: Yeah, yeah, yeah, right. It’s so important to remember and keep sight of the larger picture that you’re working with the larger team, guys are in this together, you’re taking small steps and making progress as you go along. And as you guys shifted a lot of your marketing towards mobile, I imagine just a lot of the tactical elements of marketing strategy had to change – as you mentioned, one: you had to work with an MMP. Right, and that wouldn’t even have been necessary when you were only on desktop. What were some of the other elements of the tactical execution or overall strategy that had to change as you guys oriented a lot more of your marketing towards mobile?

Holly Chen: I would say mobile is still the secondary, a complimentary to desktop. It didn’t become the major channel because desktop is still our main channel. It is still the higher quality of the two segments. And

We shifted a lot of our efforts from purely top of funnel top line growth to really understand the cross device impact. So for example, if someone downloaded a desktop and didn’t have a mobile mobile app installed, traditionally we only had email or in app notifications, and we started to experiment with the retargeting efforts to these new existing users. So over time, we did adjust our approach.

And two is, like you said creatives is really important. Initially, we were just using the desktop creatives and just, you know, resized it for mobile, but we found out it is actually a mobile native, mobile first design makes a big difference to our conversion rates. So we started experimenting, you know, things like, you swipe up to download or small things like that actually makes a big difference.

Shamanth: And you would think a lot of that is intuitive. But, again, once you’re shifting from desktop, I imagine it takes some amount of stepping back and realising that this calls for a different approach altogether.

Holly Chen: For sure, we made our share of test hypotheses and half of us didn’t approve. But we really enjoy the learning process and paved the way for us to scale up over time.

Shamanth: Indeed, indeed, and again, to the extent you’re comfortable sharing, can you talk about the impact of the mobile initiative that you guys undertook of the shift that you described?

Holly Chen: Yeah. So for example, in 2017, our new team creation from organic mobile started to decline in Q3. And once we started the mobile initiative in 2018, when we compare the same time period, the team creation from organic mobile increased, and the trend correlates very closely to the timing of starting mobile paid. So, of course, correlation is not efficient. But we’ve seen third party studies to show that each paid install results in about 1.5 additional organic installs and our results are very similar to that finding. And at the same time, we observed a significant increase in our unique pageviews for App Store and also downloads year over year that correlates with the paid mobile efforts. And also, we’ve seen an increase in the retention and engagement for users who have both desktop and mobile devices.

Shamanth: Indeed, and we certainly are all aware of the kind of trajectory that Slack has been on just in the last couple of years. And this has been incredibly insightful. Getting to see what happened behind the curtain of a lot of the mobile growth that really propelled some of the last couple of years of Slack. Holly, this has been very instructive, very honoured to have had you on the Mobile User Acquisition Show. As we wrap, could you tell our listeners how they can find out more about you?

Holly Chen: Yeah, feel free to follow me on Twitter, it’s HollyNY, and I’m on LinkedIn as well and looking forward to getting in touch and continuing discussion. Thank you so much.

Shamanth: Absolutely. Thank you again, Holly. It’s been a pleasure having you.

Holly Chen: Same here. Thank you.

A REQUEST BEFORE YOU GO

I have a very important favor to ask, which as those of you who know me know I don’t do often. If you get any pleasure or inspiration from this episode, could you PLEASE leave a review on your favorite podcasting platform – be it iTunes, Overcast, Spotify or wherever you get your podcast fix. This podcast is very much a labor of love – and each episode takes many many hours to put together. When you write a review, it will not only be a great deal of encouragement to us, but it will also support getting the word out about the Mobile User Acquisition Show.

Constructive criticism and suggestions for improvement are welcome, whether on podcasting platforms – or by email to shamanth at rocketshiphq.com. We read all reviews & I want to make this podcast better.

Thank you – and I look forward to seeing you with the next episode!