It generally comes from some user feedback or industry data pointing we’re not doing as well as we can on a specific metric. Sometimes, it’s just a guess from someone, which is generally based on his experience.

To avoid bias, we generally do a simple split (50% treatment, 50% control) on all users. 

For some experiments, we can choose to target a specific user segment (e.g. new users or power users) depending on what is the objective/problem we want to solve (e.g. increasing the success rate of newcomers, which is lower than existing users).

Me, my head of growth, and some marketing folks. Decision was in the growth team hands, who was in charge of this landing page. 

Time to develop: 2 days to build the landing page and do some back and forth validation. Then 2 weeks of test, and 1 day to analyze and share the results.

For this experiment, visit to sign-up conversion was pretty obvious. We considered looking at other metrics like average time spent on page or going deeper at the bottom funnel but we did not finally choose them for reasons like tracking too complex or relevance.

For my team, this overall structure is a common way for us to do experiments.

Pierre-Jean Hillion

This experiment framework helps us to save time on the design, precisely understand outcomes, and easily share results.

Improve main landing page CVR by adding verified reviews above the fold

capitalizing on reviews of our customers.

User tests we’ve done over the past weeks outlined the 

need of our users to be reassured to directly sign-up

, making us think verified reviews are a great first step to do it. In addition to that, 

38% of website visitors say they lack confidence into the product.

Our current CVR on this page is aligned with market average but not standing out, making us feel we can do better.

Add to our main landing page a section with reviews from our top 3 most known customers. This section will be visible above the fold both on desktop and mobile, to tackle as fast as possible users lack of confidence.

To test this new landing page version, we’ll create an A/B test.

 this new version test to 50% of the landing page audience (treament), without specific targeting criteria, to avoid specific channels conversion bias, and maximize volumes for better data quality. The other 50% visitors will be part of the control group.

 (currently at approx. 8% over the past 3 months).

 Don’t decrease activation rate of users signing-up through this landing page.

Data team bandwidth is currently limited, so we’ll have to set-up the A/B test and analyze results on our side, making the experiment building and analysis a bit longer than expected.

No direct cost associated to this experiment.

 (visit to sign-up) of the new version of the landing page 

 (12% of conversion vs 8% for the initial landing page). We landed 

, which was 11% of conversion, validating the impact of adding social proof to the landing page.

, making us feel users were overall most engaged on the page thanks to the customer reviews.

 of the users who signed-up from the new landing page version 

 to other users, meaning sign-ups quality did not decrease.

Considering the success of this new landing page version conversion wise, our 

recommendation to deploy it to 100% of our audience

We should keep monitoring user feedbacks on their level of confidence towards the product after that, considering it was one of the main problem we wanted to tackle with this experiment.

Landing Page Conversion Improvement at BlaBlaCar

Improve main landing page CVR by adding verified reviews above the fold

1# Hypothesis

2# Experiment description

3# Success metric

Sign up to see more