The Growth team wanted to introduce a free trial to help improve our signup-to-pay conversion rates. We had a ton of user feedback about this approach, but actually creating a free trial felt like a costly one-way door and a test in an earlier Reforge world didn’t perform super well. We decided to test this idea in a lower-lift and reversible way by allowing prospects to preview select pieces of content behind our paywall. While not a full trial, we believed this preview set would give sufficient context and get prospects excited to purchase a Reforge membership.

Because we didn’t have an AB testing tool or sufficient sample size on the timeline needed, we just looked back at performance of users who engaged with this feature set vs. those who didn’t, and compared their down-funnel conversion rates. While imprecise and subject to sampling bias, the impact was large enough that we felt confident this was improving conversions.

This was a huge win, primarily because we got strong signal on our overall hypothesis and the lift to do so was low.

These are two Reforge content types that were shorter and more actionable. Though less popular than our core programs, these types of content drove higher conversion rates to membership than the more popular content types.

This was surprising — we historically catered to senior PMs/growth leads and above. We didn’t make major changes, but this did validate that investing in more junior programs helped drive stronger acquisition.

Sticking with this low-risk approach helped us make a stronger case for the larger swing.

Alexis Johnson-Gresham

Ginny Mineo

This document summarizes a collection of experiments we ran to improve sign-up to payment conversion by showing previews of paid content, which at the time was the highest priority for my team.

Since mid-June, we added previews of programs, lessons, projects, and concepts to the dashboard for applicants and accepted users.

previewers had a +[redacted]% apply to pay CVR

 compared to users who didn’t view previews (XX% higher for accepted previewers vs accepted non-previewers).

The most popular resource is programs, and the top converting programs were Growth Series and PM Foundations, potentially signaling a widening of our audience to more junior professionals.

We expect to continue this track through exploring a free trial.

Increase the conversion of accepted users by driving them to previews of content behind the paywall (programs, lessons, projects, and concepts).

Added previews of programs, lessons, projects, and concepts to the dashboard for applicants and accepted users.

When users clicked on content that’s not available to preview, they saw different paywall messaging:

Adding badges/copy across the signup experience to signify that previews were available.

Removing auto-popup of the payment modal upon page load (direct links to the payment modal still worked)

Users who viewed a preview had a +XXpp apply to pay rate compared to users who logged in but didn’t view previews.

Previewers had a ~YY apply to pay CVR compared to users who didn’t view previews (Z% higher for accepted previewers vs accepted non-previewers). 

Sources: apply → paid / accept → paid, 8/1 - 9/30 for apps submitted since 8/1

Top resources by volume since Aug 1 are primarily Programs, and the more “entry level” programs tended to have higher conversion rates:

Mastering Product Management (XX uniques)

 have a higher CVR to paid than other resources.

What have we learned about our original hypothesis?

There may be some selection bias (more engaged/qualified users may seek out resources prior to converting), but we believe these previews have been a net positive on CVR to paid.

Programs continue to be the bread and butter of our business in terms of volume, but we may be underinvesting in using concepts/projects to drive home the value of Reforge to members.

Based on popularity and stronger CVR of Growth Series/ PM Foundations, 

we may be skewing to more junior audiences compared to ~a year ago.

What are we doing next as a result of these learnings?

Better understand how valuable these resources are. We’re exploring some qualitative research to better understand how useful these are.

Use these learnings to inform the free trial motion and potential quick wins in signup → pay.

Paid Content Preview Analysis at Reforge

📋 Executive Summary

What was our original hypothesis?

What did we do?

Sign up to see more