---

About Ekaterina : LinkedIn

FULL TRANSCRIPT BELOW

Shamanth: I’m excited to welcome Ekaterina Gamsriegler to Intelligent Artifice. Ekaterina, welcome, and welcome back to the show.

Ekaterina: Yeah, it’s great to be back. Thanks, Shamanth.

Shamanth: Yeah. Certainly, I’ve followed a lot of your work for a very long time. Been very impressed by both how clearly you’ve taught, but also how you have actually made meaningful differences to the products you have worked on. Also, just when I talk to you, you talk a lot about some of the trade-offs and scenarios rather than just hacks, which I’m truly impressed by. What’s your brief answer when somebody’s asking you, “What do you do as a growth lead?”

Ekaterina: Yeah. The answer has been evolving over the last years, but I think now, even though I still struggle with putting myself into a certain box, I would say that I optimize the whole chain. I optimize the systems end to end and make sure that the revenue or the investment that we do on top actually exits also at the bottom.

So yeah, hacks and busy work might not exactly be my most favorite things. For me it’s about pursuing a diverse set of bets which are more likely with a higher level of confidence going to make an impact than not. And yeah, optimizing the whole chain from acquisition to renewals and making sure that it works.

Shamanth: And they are all interdependent and they all involve humans. It’s not like it’s a math puzzle. And you talked about some of the interconnectedness between a lot of these pieces in the past and also when we were preparing for this call. Talk to me about how and why it’s important to think about this as an interconnected system. And also, was there a moment when you actually saw the whole picture of the interconnectedness between all of these pieces?

Ekaterina: Yeah, I think the reason why I switched to, again, a bit out of the box thinking when it comes to compared to just working on acquisition marketing, on which I’ve spent the first decade of my career, I think about four or five years ago my thinking started changing a lot because I’ve lived through a lot of successes, but also through a lot of nightmares, which might be familiar to so many people working in growth, especially on the marketing and acquisition side.

Because very often we discover that there is an experiment with the paywall running which we were not aware of. We discovered that when there was the paywall, the call to action button was redesigned, maybe some team forgot to put the event tracking into it. Something changed, something broke along the way, breaking the conversions and breaking the acquisition ultimately.

So this is where four, five years ago, I also was launching my own course. I was starting to consult a bit with a few clients, smaller clients here and there. I started mentoring very extensively and had more than 100 sessions on different platforms. And ultimately, I joined Mimo, where I actually had the chance to have an impact on what’s happening on the product side.

Being exposed to multiple challenges, multiple cases, different verticals, and hearing the same stories over and over again made me realize that yeah, there are things that are definitely so much more connected and should be monitored much, much closer. So this vision for full funnel growth end-to-end analysis crystallized so much better than what I had before.

Because even if everybody’s on the same page and if there is a shared understanding that the teams should work together, there is no product without marketing and the other way around. I think it’s still very difficult to live in the same context and with the same views because even our dashboards are often different and have separate views.

Shamanth: Sure, for sure. And the different dashboards thing is something that in theory feels like it’s an easy problem to solve and it never is. Is there an example that comes to mind of the interconnectedness going wrong or going right that you can think of?

Ekaterina: I think my most recent examples from the past years are surprisingly about hitting the growth ceiling, but not in the sense that we cannot scale acquisition with good returns anymore, which is also a shared challenge. But actually situations where you see that the amount of customers and subscribers churning exceeds the number of customers that you can bring in.

So this happens to products at various stages, and I think this tends to hit me very hard because so far I was usually under the impression that, okay, churn is a big problem, but this can be fixed once we fix many other more important things at the top of the funnel.

But I worked with a few businesses where it was actually a very urgent problem to solve. And usually this would also happen after a few very successful months when it comes to acquisition. Churn would follow, and the math would not work out. Other cases were when the funnel would look pretty decent and scaling was possible, but at the same time there was a deeper problem with aligning the value of the product to the target segment or the value proposition that we had.

Solving this through marketing and advertising was also very interesting and again, made me realize how much impact telling a certain story has on the whole product metrics.

Shamanth: For sure. For sure. Yeah, because if you’re not telling the right story, nothing you fix on the product might work, and vice versa. Or if the product is broken, nothing you do on the marketing will work. So from what I’m hearing you say, you really need to look at both those pieces of the picture and to really work on both those and not just operate in the local maxima.

Ekaterina: Yeah, exactly. Yeah.

Shamanth: Yeah. And you also talked about how you typically have a scorecard for every experiment that you run. Walk us through what that scorecard looks like in practice when you are trying to ship a new feature, concept, anything at all?

Ekaterina: Yeah. When I joined Mimo, I really liked the format of pitches that we had for literally any experiment or any feature, any new advertising creative concept, which everybody had access to, which was very transparent and everybody could contribute.

This originally was from the book called Upwork, I think by Basecamp. And it has a lot of many more valuable artifacts in it. But what I did is I added a small section which was supposed to say what are the primary metrics that we are planning to move?

What are the secondary ones that might get affected? What are the trade-offs? If this moves, what can be affected negatively? And the guardrails in terms of what are the limits that we’re willing to go to and what we are definitely not willing to risk. So this also helped, and it was not about making it harder to pitch something, but it was about understanding the possible trade-offs.

And I personally find this exercise very valuable for everyone because over time, if you’re a good product manager, if you’re a good growth professional, you also know many ways of how to manipulate a certain metric.

And I also think over time, the teams, companies, people on the teams can really find ways to start abusing different metrics, which might happen for various reasons. But I think that this exercise is very valuable because it helps you truly understand what are you trying to move at the expense of what, and what might get affected along the way down the funnel.

Setting the guardrails is also making it much harder to manipulate the narrative after. Because I remember in the early years of my career, I’ve seen it quite often when people would start formulating the hypothesis after the experiment results were in, so just using the data to fit the narrative or the assumptions that they had.

And I think this exercise again prevents you from that. It holds you somewhat accountable for being honest. For example, it’s easy to say, let’s test a certain trial duration, let’s make the trial longer, without long-term looking at the cancellation rates or trial to purchase rates ultimately.

Or I remember the case when we changed the design of the paywall and we made it very clear what are the premium benefits for different groups of our learners, personalizing them based on the answers they would give on onboarding. This did not have any impact on the trial opt-in rate, but this had a good impact on the cancellation rates because once this would resonate with users, they would upgrade and they would be more likely to not cancel the subscription when they understood how these premium benefits actually fit into their goals.

The mistake I often see is teams saying, “This is our target metric, and these are going to be the 10 experiments that we’re going to run this quarter that will move this target metric positively.” And while I understand the importance of this from the experimentation perspective, I also think not looking at three to four metrics and broader context at least, or projecting, the hardest is ultimately the LTV-based revenue, not only the short-term revenue, but the LTV-based revenue. So not looking at any of this can be a big mistake.

Shamanth: Yeah. And I 100 percent agree that if you’re fixated on one single metric, that has a lot of unintended side effects even if it keeps everybody laser-focused on one thing. A long time ago, I actually wrote this piece on how there was a somewhat well-known case on Uber and how I think one of their vendors had a lot of fraudulent traffic.

And my point was, it was really because of incentives at Uber, because the team was incentivized on day one and day seven metrics. The team was incentivized on how much they’re spending, and nobody really cared about questioning fraudulent traffic, because the fraudulent traffic actually hit the D1 metrics, D7 metrics, and that was the whole point of the fraud, that it had to look genuine. And so I think that cost them multiple millions, which is kind of crazy.

And I think what you’re saying has the same point. Even if somebody doesn’t intend to be malicious, it can have unintended side effects that nobody’s anticipating.

Ekaterina: Yeah, even this, I think, without intent, a lot of these things are very easy to lose track of when you are not even thinking that this might be the consequences. But yeah, it’s much worse when you are aware of the consequences and by manipulating one metric, by increasing it, you just do it in ways that do not really move up the metric that is a level higher than that.

Shamanth: Sure, for sure. And oftentimes, if you’re working in growth and marketing, you make decisions that you don’t see the results for multiple weeks. If you’re working on the acquisition side, you have an immediate result in the dashboard, but there are also downstream results which you don’t see. Are there examples you can think of where a team’s making acquisition decisions that cause damage that shows up many weeks down the line?

Ekaterina: Yeah, I think there are some extreme cases that I’ve seen. Some would show up weeks down the line and would be hurting the revenue ultimately. Some would crystallize, materialize over the years and be hurting the perception of the whole brand. So this is also very difficult to turn around.

For example, I remember that when I joined Mimo, we had a lot of really great creatives. I personally really loved them. They were these very cute animated characters, typically. A lot of static images too. We even had an ad where an animated cartoonish character would reference a person sitting on the toilet and learning how to code, and this was working really well from a top of the funnel perspective.

Over a few months and years, it started becoming clear that the perception of the whole product was more that it is a game, that it is for kids. So this is what we would often read in user reviews. A lot of friends or industry folks would come and say, “Oh, actually my kids are playing your game, the game that you’re working on.”

While all of this was fun and cute and it was bringing and unlocking certain amounts of revenue, it was clear that this is not exactly the brand perception that we wanted to have because we had very deep, very high quality content in the product. And there were a lot of success stories from the learners who learned to code with our app and changed careers.

So it was becoming harder to actually make it clear that we have a very serious product with a lot of content to go through that you can use to boost your career and skills. So yeah, this was one of those extreme cases where over time we also did a rebrand.

But as I mentioned, advertising is also a way to tell whatever story you want. I earlier gave an example of the product where the core value would be great, and the product would be great. At the same time people would not be willing to pay for it because the audience was too broad, the use case was too broad.

And only after we decided to pivot to a more narrow segment we started seeing much better returns, after reshaping the value proposition, adding more features to the product for the specific segment, redoing the store listings and all the creative concepts. This is when we could truly unlock profitable acquisition by retelling the story, basically, after telling a different story for the years prior.

Shamanth: Yeah, I think another…

Ekaterina: Go ahead.

Shamanth: Yeah, I was just going to say that’s crazy that even though the cute kid creatives probably gave you very good metrics…

Ekaterina: Yeah, they were, yeah.

Shamanth: …they brought in completely the wrong kinds of users. And you would probably not see that impact in any user acquisition dashboard. You would see that impact in the product retention metrics, which is scary when you think about it.

Ekaterina: Yeah, I think even if you would look at the funnel, you would be really okay with it because there was no specific bottleneck or problem to identify. It’s more that it was popping up from all the conversations with users and the qualitative user research where we would be like, “Is it really how we want to be perceived? No, actually not. We want to be perceived as a more professional tool.” And this is what was triggering it because yeah, over the years we would probably hit a ceiling in positioning ourselves the old way.

Shamanth: Yeah. I can imagine. Yeah, that’s crazy. And are there similar examples you can think of on the paywall and monetization side as well?

Ekaterina: Yeah, I think on the paywall and monetization, maybe the most dangerous tactic from my perspective, which I’ve also seen backfire a lot, either in the short term, like within a few months, up to longer term, within years, is discounting during seasonality.

Because typically you see very good growth at the beginning. The smart way is to immediately watch out for the cancellation rates and refund rates, like on day zero to day seven, for example, to see if the users who are subscribing for monthly or yearly purchase subscriptions are canceling right away or if they’re staying with you.

So I think this is a good way to at least keep an eye on that. And also discounts that are aggressive and frequent, discounts for everyone instead of discounts that are more segmented, because as we also talked before, it gives you a boost in revenue.

I’ve seen it multiple times that it’s very easy to get addicted to this revenue boost that you are getting every month or every few months depending on the frequency, and it’s also not easy to get off this needle, so to say. It becomes like a real addiction. It’s not that easy to revert this trend.

So overall discounting, whether it’s about making sure that people are not canceling right away or that this is not negatively affecting your lifetime value in the long run and the way you structure your growth engines, I think this is very important to watch out for.

Also, an interesting question is how do you do discounts? Do you offer the discount that stays for multiple renewal periods, or do you offer it once and then charge the full price? I’m actually curious, how do you usually do it, or do you have a take on that?

Shamanth: Yeah, transparently, while I’m not involved directly on the product side, I have partnered with different folks. There are at least a couple of clients we’ve worked with where they said, “We want to give discounts. We want users to perceive us as the Walmart because that is a retention lever.” Every time we offer a discount, people will come back. It’s effectively CRM for us, right? And they’ve been very intentional about it, and it has actually worked well.

But there are also products that I work with that are on the flip side, like you mentioned, where users just get used to the discount. They wait for the discount, they expect a discount, and that is a net negative. I’ve certainly seen both those extremes, and I definitely understand that it’s a challenge if you just treat the discount as a short-term fix.

Ekaterina: Yeah, and I’ve also been dealing and working with different teams that have different attitudes to that. I think there are teams that think we don’t need to do discounts at all because our product is great, and if we do it, we will offer it once and then renew at the full price.

There are other teams, and I fall into this category myself, where I want to be transparent and honest about the discounts. So if we’re doing it, then we also allow the renewals to come with a discount as well. And I overall see that this approach tends to bring more recurring revenue.

But there are so many elements to it. There is ethics, there is the perception in the team about how it should be done, then there is the risk of getting dependent on it, there is a risk of cannibalizing your revenue, or maybe testing if it brings more recurring revenue in the long run, and what impact on LTV it ultimately has.

So yeah, lots of things to consider, even if it’s just a very simple promo campaign where you give 50% off.

Shamanth: Yeah. Yeah. And which also makes me think, a lot of growth problems can often just be team dynamic problems. As much as they are about understanding users, you had to manage the dynamics within your team. So how do you have some of these conversations within a growth team without it just becoming finger-pointing?

Ekaterina: Data helps a lot. Looking at the facts helps a lot to avoid speculation, right? That’s why I think it’s always important to have the weekly check-ins or biweekly check-ins where we take an honest look at the funnel, at the performance of the new cohorts. And at the same time, we have the log of changes, which is documented, where we document what has changed on the acquisition side, if there are new channels that are tested, if there are new creative concepts, if there is a new optimization event that we are experimenting with.

So all of this is about defaulting to transparency, and everybody being able to always trace back and look at the numbers that we are all looking at, instead of one person pulling the data from three sources, another one pulling from three others, and then discussing why they do not match.

Also, I think it’s about a very firm belief which I have with the teams I’m working with, that nobody is ever coming on a certain day to intentionally do a bad job in any sense and to launch a creative which is misleading. This is never the mindset that I, in my 15 years of a career, I never came across a person who would be doing that.

Usually, everybody’s really trying to do their best job, right? Coming from this perspective I think helps a lot too. But ultimately having the numbers, having the log of changes where you know what exactly happened and what might have contributed to a drop here and there.

Also getting the qualitative insights. Again, having the evidence of, okay, under this video, these are the type of comments that people are leaving. And now maybe we have an influx of support tickets which address this and this, or maybe we did the partnership with an influencer where they mentioned that the product is free, but we actually have a free trial, so now everybody’s complaining in user reviews that it’s not free, it’s a scam.

Don’t do that. So having all of this evidence on top also helps to make sure that we are really going to the right audience with the right messages and the right value propositions. Because I think it’s one thing to define your value proposition in a vacuum and to say, “This is what we stand for. This is what we believe our product is for, and this is who it’s for.”

And another thing is to be relying on a weekly basis on this multitude of data points which actually tell you if you’re on the right track or not. So I think it also changes a bit the mindset from pure growth and performance objectives for paid advertising to seeing it also as an investment into learnings.

Because ultimately, we’re also learning a lot about our target audience and about our potential users, and this requires some investment too.

Shamanth: Yeah. Yeah. And it’s never easy or straightforward, but that’s why it is supposed to be hard.

Ekaterina: It’s a process, yeah.

Shamanth: Yeah, it is a process like everything. Yeah. Again, when you’re working on a completely new project, what are some of the first things that you look for to see if the dominoes are about to fall?

Ekaterina: Yeah, I might have also become pretty biased in the last years, but my go-to first thing to do, no matter what I’m hearing, usually is the analysis of the funnel. It’s the analysis of how the monetization system works, which is not just about how the paywall looks, but it’s about long-term trends, looking into the seasonality, how things have been changing and evolving over the months and years.

But then also taking a good look at the full user journey from ad impression to renewal and understanding if everything truly fits and where the biggest problems and misalignment can live. Because I keep coming to cases where the problems are in the value breaking somewhere, and this can start already at the acquisition stage, but it’s rare.

It most usually is somewhere at the activation and conversion stages, and this is why I always just start with looking into the funnel. Now I also take a deep look at the MRR and active subscriber movements to see how big of a problem churn is. But this is more about prioritizing what kind of actions need to happen first and what is the most urgent problem to solve.

Shamanth: Oh, for sure. Yeah. And to switch gears a bit, AI is something a lot of people are plugged into, quite the buzz lately. So what you do is very cross-funnel. This is exactly the kind of thing you couldn’t just go into an LLM and ask answers for, at least not in a way that’s very straightforward without a lot of product-specific context. So where, if anywhere, do you see AI fitting into the kind of cross-funnel analysis that you’re talking about?

Ekaterina: I think there might be more applications of it in my work than I might be aware of at the moment. Also, we talked before that I feel quite behind when it comes to that. But what I don’t see myself or also the teams that I’ve been working with using it for is necessarily ideation.

I think there is always a huge backlog of ideas, and it does not matter if we’re talking about creative concepts, new ones or iterations, or if we talk about paywalls or new features. Ideas are plenty. Usually when I work on something and once the problem is clear and identified, I also have a lot of solutions already there.

So I don’t necessarily use AI for that, but what I use it for is to challenge my thinking and to poke holes in my solutions and proposals. Because this is where, the more cases you’ve seen, the more you might be stuck on a certain way of doing things or even trying to follow a certain playbook, and this is where I usually use AI to challenge me.

And of course I have a lot of applications for it at the top of the funnel, so not necessarily a full funnel analysis, but creating new concepts like AIGC, iterating on the concepts, using it for textual metadata optimization for the stores, translations, stuff like that.

Shamanth: Yeah. Yeah. And you’re right that using it to really challenge your thinking is just so critical because I’m still astonished. It’s been a year and a half or so since I really started diving in, but I’m still astonished at how it continues to give us ideas. That’s something I’m building right now, which is a creative intelligence tool, and it just came up with ideas and paradigms I really could never have thought about. And that’s certainly something that’s astonishing to me still.

Again, to switch gears, if you had to look back, you worked on numerous products for a very long time. What’s something you got wrong early on that shaped how you think about funnels today?

Ekaterina: I think not maybe catastrophically wrong, but what I’ve been way more reflective about in the past years is asking myself where these original beliefs are coming from. And I think there are two things which I tried to fix in the middle of my career once I saw that it’s really not working.

So the first was how I see creative production and how I work with creatives, because in two different companies, I used to lead the creative production efforts, and I truly loved it. It was amazing to work on this. They were very talented teams, very good ones who were super passionate about what they were doing.

And we would look at the past performance. We would think of how we can iterate on a concept or come up with something completely fresh. So we were using data. We were, like, halfway there. But what I’m realizing is that we were very far from doing user research. We were not talking to users.

We would only have glimpses of what they think judging by their interaction with the ads and maybe writing comments sometimes. So this was a big missing piece for me because we would have amazing concepts, and I’m proud of them still to this very day, that the teams would produce, but they would not resonate.

Using user research for creative production has become for me a must in the past years. And also a similar attitude to data. Because for the first half of my career, I was often ending up in companies where I would look at the dashboards, analyze every cent, but still not necessarily go too deep into whether these assumptions were true.

And I remember one of the extreme cases was when we were experimenting a lot with driving more users to the yearly plan. We would also try to do everything to get more yearly purchases in, before we realized that the lifetime value of the monthly plan was actually higher than the yearly plan, which happens sometimes apparently.

But taking somebody’s word for granted and just operating with these assumptions for months and years, this is when I figured okay, I also need to get better at data analysis. I need to be able to query everything myself to double-check everything if needed. So I think again, this maybe still holds to this very day: user research and being fluent with data and being able to analyze everything yourself is what I stick to still.

Shamanth: Yeah. Yeah. I think the more patterns you see, the more wary you are of the exceptions like that, where the monthly plan has more value than the annual one, which you wouldn’t expect.

Ekaterina: Yeah.

Shamanth: Until you do. And I think there are others. A similar exception I’m noticing is the economics of AI-driven products where a free trial is no longer free. You are basically looking at the cost of everything. That sort of thing you need to be very careful about, revisiting your assumptions.

Excellent, Ekaterina. I think this has been incredibly instructive, as always, as every time I’ve spoken to you. This is perhaps a good place for us to start to wrap up. But before we do that, could you tell folks how to find out more about you and everything you do?

Ekaterina: I’m trying to write more on LinkedIn to share more, and this is probably the best platform to follow, unless I finally get to start my own newsletter.

Shamanth: Excellent. We will link to your LinkedIn then. Yeah. And now this is a good place for us to wrap. Thank you so much for your time.

Ekaterina: Thanks, Shamanth. It was a pleasure. Thank you.

Shamanth: Yeah. Excellent.