When I’m prioritizing experiments, there are a couple of things that I generally look at. One, of course, is the impact: the overall exposures, how many people get this experiment, and the overall lift I expect it to drive. The other is how quickly we can collect results from it. In this case, someone clicks to sign up to Webflow, we open a modal, and then they go through the sign-up flow. Even though this already had a decent conversion rate, the experiment runtime wasn’t very long, so we could collect results very quickly and start to iterate on this. Since we thought that the lift we could generate in our treatment group was going to be pretty high and the runtime was low, I decided to prioritize this over some of the other experiments we would’ve been able to do at the time.


We could have gone super in-depth on user stories and done a lot of research, but at the end of the day, it’s a sign-up modal, right? It’s not a massive new product or a completely new feature, so I knew that we needed to hit the right balance between how much information to provide and how quickly we wanted to move.


I think one thing that people don’t really consider as much as they should when they’re planning experiments is that where you’re experimenting is just as important as what you’re experimenting on. For example, where you’re experimenting within the funnel can make a big difference in terms of time.


It can be hard to know when to stop experimenting. I think it comes down to recognizing when you’re just validating the same hypothesis over and over again and not getting any new signal. I try to move forward and push the team into different areas before we get to that point. If you’re working at a huge company, like Meta or Google, maybe those small optimizations make sense, since even a minor improvement might result in millions of dollars of revenue. But if you’re working at a startup, most of the time you won’t have the data to dig in and over-optimize areas, you need to focus on bigger bets.


We ended up shipping the experiment and it was a win; it brought around a 12% increase in sign-up conversion rate, and we’re still iterating on it. Maybe the bigger learning though was that, from a top-of-funnel perspective, the more we show our product functionality, the more it helps users get into the right mindset to build a website. It’s something that we’re still experimenting with across many different areas of our funnel and sign-up flow in general.


I start to involve designers once I have the context, problem statement, hypothesis, and prediction in place. On this project, that let me go to them and say, “We’re seeing this decrease in conversion rate, can you explore different designs that you think would improve it?” One thing that I haven’t always done in the past is give clear constraints. Without those, I’ve found that designers can kind of go all over the place. Giving them the right level of constraints can help make sure that they’re focusing on the right growth levers.


On this project, we just brought in an engineer and a Webflow designer. Since our whole website is built in Webflow, this just required small snippets of code, so the work here was mainly designer-driven. They were the ones flagging things that we should think about from a technical perspective.


Bruno Estrella

Low-lift experiments, like this one we did at Webflow around our sign-up modal, can still deliver significant results if they’re run well.

Asana task | Jira epic | Figma file | PM: Bruno, Design Lead: [redacted], Eng Lead: [redacted]

The main action we promote on our website is the call to action to sign-up

. Today, [redacted]% of people who click on the Get Started - It’s free button end up signing up to Webflow.

When we look at it from an absolute number perspective, we have an average of [redacted] visitors clicking the CTA to sign-up per day, [redacted]/month, and [redacted]/year. It’s interesting how [redacted]% of people don’t end up signing up even though they showed a significant level of interest in our product.

 A [redacted]% lift in sign-up conversion rate means an incremental [redacted] sign-ups/yea

The main problem we’re trying to solve is to

 improve the sign-up conversion rate after users click the CTA button to sign-up.

We believe visitors are still not 100% convinced to sign-up to Webflow after they click on the CTA button. 

Evidence: [redacted]% drop-off rate from clicking CTA to sign-up

If we add extra validation elements in the sign-up modal, visitors will have an extra motivation boost to sign-up for Webflow and try out the product.

All solutions used by example companies in their sign-up flows (see “Examples” section above) sounds like a low or medium engineering effort, nothing big.


As long as the content is static, it’s all about arranging it on the signup page, honoring various viewport sizes

Effort increases if we want to load content dynamically, for example if we want marketing team to be able to easily update logos, quotes, etc. without a need to ask engineerings to perform the update and without a need to wait several days for the update to happen

Small effort, but might be tricky to do it right

Sounds like replacing white background with a static image of the Designer app for example. With all the care of a variety of viewport sizes.

Sign-up Modal Redesign Experiment at Webflow

Sign-up modal redesign experiment

Context

Problem statement

Hypothesis

Sign up to see more