Dan Wolchonok

This launch analysis helped the Reforge team and I gain insight into weekly active users, login rates, and conversion after launching a new sharing feature aimed at driving product growth.

we set a goal of delivering $500,000 in 2019

. The key variables for us to deliver on this goal:

The percentage of weekly active users that invite guests

The number of invites per weekly active inviter

The purchase rate for accepted applicants

 that contained the input variables to our success goal:

Before we launched the feature to the user base, we emailed users individually to tell them about it and waited to see how they invited others. After we saw that people weren’t confused and that the usage was what we expected, we broadly promoted the feature to the user base.

We evaluated the performance of the feature in the 90 days after it was broadly available.

We looked at the number of people who invited others as a percentage of users looking at content

. We looked at the ratio of people who invite guests to users looking at our content. The ratio ranged from 5-10%.

 users who didn’t invite any guests and they said they weren’t aware of the feature and that they would be more likely to invite users in the future. There was a noticeable increase in the ratio as we entered our fall cohort and we marketed the feature and encouraged its use with people enrolled in our fall programs (October 2019).

While we didn’t hit our 10% goal in the first 90 days after the feature launch, we feel confident that we can increase this rate given more time and additional surface areas to encourage guesting. Examples could be when a user creates a bookmark or logs back in after a period of inactivity.

Our goal was 1.15 invites per inviter, and we saw an average of 1.71 invites per inviter.

Just because someone invites a guest doesn’t mean that the person receives the email, opens it, clicks to open it, and successfully logs in. 

It’s important to understand whether users are making it through this funnel and dive into it to see if any part of it is performing poorly

Overall, the weighted average of the login rate for users is 77%, higher than we had anticipated.

We predicted that 33% of guests would submit an application, and we observed a weighted average of 36% of guests applying to Reforge within 90 days of their guest invite.

We predicted that the acceptance rate for those applications would be 70%. The weighted average was 53%. This was lower than we anticipated and we need to think about whether we should relax our application acceptance criteria in situations like these.

We predicted that the purchase rate would be 60%. The weighted average was 76%.

There were three main concerns with the launch of this feature:

We didn’t see guests copying our content at higher rates than existing members

2. It would be a mechanism to avoid paying for access

We saw healthy rates of net new customers purchasing

We saw the vast majority of users only get one invite as a guest. In qualitative feedback, a lot of people didn’t actually get to log in and access the material in their first invite and the second represented the first real access

3. It wouldn’t be material to our business

We set a goal of $500k in revenue for the year, and in the first 90 days, we were well on our way to exceeding that goal based on meeting or hitting most of the input metrics.

While the data demonstrates that it’s a successful launch (and has the potential to grow as our customer base grows), there are clear areas for improvement from doing qualitative and quantitative analysis of various pockets of users.

Guests often didn’t know why their co-worker had invited them to the material (we don’t allow them to send a custom message in the invitation)

Guests didn’t always have time to login in during the 7 days between the invite being sent and their access expiring.

Even if guests did log in, they frequently had to ask to get access again as they'd didn’t have time to sufficiently digest the information shared with them.

Of those that log in, 80% are active for 2 days or less in looking at the material.

Of those that log in, the majority of users look at less than 3 lessons in a module (most modules have 10+ lessons).

The qualitative and quantitative data backs up the initial hypothesis that this feature is not a way for prospective customers to avoid paying for Reforge, it’s a mechanism to help our customers get buy-in from their colleagues. It is a great area where the interests of Reforge customers and Reforge are aligned. It is a big value add for Reforge as it helps to accelerate the spread of our material and brand inside companies. The opportunity is in prompting people to increase the invite rate, getting guests to login, consume material and understanding why they would benefit.

New feature launch analysis for viral invite at Reforge

Guest Loop Launch Goals

Sign up to see more