The 'Experiments Documentation' is a concise report for each completed experiment that we run in SelectHub. It offers a brief description, relevant data, outlines the learnings obtained, and includes an overview of the criteria used for concluding the experiments. This document serves as an essential reference, enabling quick insights and understanding of the outcomes from various experiments.


Basic guidelines for declaring a winner in our experiments.


We wrap up experiments at the end of business weeks, never midway through. Sometimes we might even let an experiment run for a full month, especially if we notice that user behavior varies during different weeks.

The experiment's ID and title that we reference in various documents.


I'll connect this to our Experimentation Scorecards, where a single slide will showcase the control vs. variant along with a quick summary.


It's pretty rare for us to skip a secondary metric, but when we're zeroing in on a page with just one specific goal, it makes sense to go this route.

We examined our current user count and conversion rate to figure out what we'd need for a tangible uplift on this landing page. We opted for more substantial changes to expedite results and increase the likelihood of achieving statistical significance. For other landing pages with different needs, we might have simply adjusted the form steps and left everything else unchanged.

Ultimately, the bottom line is how this experiment will influence our revenue.

We toss this info into the Scorecards too, because it's a quick and easy way to get the gist without sifting through other numbers.

We mainly focus on Probability Density when checking out the stats, but we also keep an eye on date range performance.

This is one segment that we care. In experiments, the depth we go into the data matters. I pick the level of detail based on the impact size and the number of users involved in the experiment.


Farid Asadi

For each completed experiment we run, we create a concise report that enables team members to quickly understand the key insights and outcomes.

Landing Page Layout Experiment Documentation

Having additional metrics will help out in indecisive situations.

Product Demo Request Page; All Devices     13 JUN 2023

After diving into the data on various landing pages, we discovered that single-form layouts outperform two-step forms. To capitalize on this, we've revamped the design and text to better align with what our user research tells us people want right now. 

 (w/ >95% confidence) in the conversion rate! This suggests that single forms are more effective, and users prefer the new copy and design. We'll use these insights on appropriate PMO pages (our gateway for lead generation), keeping a close watch on the conversion rate.

This improvement could potentially increase revenue 

Single-step forms outperformed other form types, supporting our main hypothesis.

Redesigning certain elements to achieve a more modern and updated look proved beneficial.

The impact of the change was particularly significant for 

We can extend these positive results by implementing the changes on similar PMO and PPC landing pages, thereby generating additional revenue.

Conversion Experiments Documentation at SelectHub

Landing Page Layout Experiment Documentation

CRITERIA

#1 Two Step Vs. Single — Demo

Sign up to see more