I came to Oribi pre-acquisition to launch the PMM function. In my first year, I conducted a big initiative around onboarding where we sent a physical box to each new customer. It increased engagement amazingly, so then we decided to focus on in-app. The next big opportunity was retention.

At the time, we needed to explore levers to increase retention without being dependent on R&D. I realized we had an opportunity for a quick win, so the process was mainly me moving quickly and scrappily.

From churned users especially, I got the sense that a big portion of them understood what the product did, but they didn't get how to leverage the product for their specific needs. We had lots of different personas — one might be an SEO agency, a paid marketing agency, or a solopreneur who primarily does Facebook ads. We had offerings for all of these personas, but they couldn't figure out how to get there. And I felt there was something more we could do in order to get them to feel that aha moment for their use case.

We had a very robust funnel feature where you could filter by source (organic, paid, etc.). But instead of making a video about how to use that feature, we tailored the videos and the lingo to each persona. So imagine an SEO marketer clicks the SEO video. It would say something like, "You're looking to optimize your organic traffic, so the first thing you need to do is create a funnel. This is how you do it, and now filter by SEO." These little changes in lingo increased people actually watching the videos because they felt they were tailored to their needs.

We implemented this in Intercom, using the function where you can add links that look like banners in a chat. I made the videos in Loom

We had technical limitations and couldn't get the resources to do an A/B test in-app. Sequential might make the test dirtier in some cases, but since our acquisition channel was consistent, sequential was fine in this case. We chose several features to test at first, and planned to roll it out to all the other features if it was impactful.

I don't even think we measured how many clicks, but the pop-up popped up for every user who visited the first few times, and we saw that people were engaging, so we didn't really check. I wish I had access to the data, but it was relatively big numbers.

After this test, I was hoping to bring this to all the features. There was only one where it didn't work because the feature itself was relatively straightforward, and the behavior wasn't that different between different personas. But we got acquired soon after, so our roadmap changed. My only regret is not doing this sooner. I think it's important that product teams are receptive to doing these types of things in product, and wish that were the case at every company.

Arik Marmorstein

Through a lightweight, scrappy experiment, I discovered a key insight that helped more customers reach Oribi's aha moment and meaningfully improved retention.

Oribi (acquired by Linkedin) was a marketing analytics tool that generally served 3 different personas: E-commerce marketers, marketing agencies, and other companies that didn’t fall into the first two categories. Under each category, there were organizations that grew through paid, SEO, content marketing, and more. Oribi had a broad offering, but it made it harder to get people to the aha moment, to engage, and to retain. I was the head of product marketing at Oribi.

Customers don’t know how to initially get value for their own specific use cases because there are a lot of options when using the product.

We conducted several interviews with existing and churned users. We looked at the prerequisites to the aha moment as an equation during the interview, and tried to understand, kind of like plumbers, where’s the leak:


Setup*understood the value of the product*felt the value is relevant to them*experienced the value*was happy with the results*understood how to experience it again easily*can afford/get buy-in to continue using the product

We saw during the interviews that people understood the value, but weren’t sure it was relevant to them and/or experienced it.

We had several tactics in mind, and after considering the different options (impact, resources, chances), as well as the fact we didn’t have R&D resources for this 

If we give people quick shortcuts—that is, short videos—geared towards their needs in the app itself, we’ll see that more of them find what they need, and hence are more engaged and more apt to be retained.

The videos weren’t about how to use the product, but rather 

 For example, a 3.5 minute video on optimizing your search traffic using a certain feature.

We started our test with videos for 3 features in the app: Funnels, Channels, and Reports. Here’s an example of what it looked like in one of the features:

Implemented in Intercom and Loom

We also couldn’t do a/b test, so created a sequential one.

More meaningful feature-related actions per user per week (median)

We ran the test for ~4 weeks before doing an initial analysis.

Funnels feature(median meaningful actions per user):

 the in-app enablement: 100%/233% increase in meaningful actions per user, respectively.

We also saw an increase of 25% among the same cohort from the previous period (non trial users).

Paying/trial users who clicked the in-app enablement: 50%/100% increase in meaningful actions per user, respectively.

We also saw an increase of 50% between the same cohort from the previous period.

No effect - All paying and trial users showed the same median report events, regardless of whether they clicked on the Reports in-app enablement or not.

We doubled week 4 retention for the new users who clicked the in-app enablement (please mind causation vs correlation).

In-app enablement in products that serve more than one use case need to be problem-oriented, rather than feature-oriented.

Aha Moment Experiment at Oribi

Problem/Goal

Hypothesis

Evidence of hypothesis

Test

Sign up to see more