The main motivation was to improve their activation rate.

Clubhouse was a project management solution. So what typically what would happen is somebody would sign up from a company, they would invite their teammates, they would create a project, share that project, and that would be considered an activated user once they created a project and shared it. Clubhouse wanted to improve that metric and also find ways to improve their overall trial-to-paid conversion rate. So, those were the two big things I was looking at as part of this research.

I always like to do user testing if it's an option because it just tells you so much.

I think it's one of the best sources of information about how people find the experience. So you can see things from analytics. You can see that, you know, there's a big drop off point after people sign up or after they confirm their email or whatever. But you don't necessarily know why that's happening until you actually have people go through step by step and talk out loud.

. I've found that it's pretty similar overall. It's a little bit less feature-rich, but the reason I usually use it with clients is that User Testing is a little more of an enterprise product. 

Both have similar targeting capabilities to define your target audience, and you get results pretty quickly. You can then summarize the findings and create clips within Playbook.

These were unmoderated tests. So basically the way it works is... 

, they have testers that have signed up to be testers on their platforms and get paid to do these tests. And they have a Chrome extension or something where as they're going to your website, they're also seeing each of the tasks, and their screen is being recorded.

And so you, you just set up the test. It gets sent out to their panel, people respond to it, and then usually within an hour, you have all your responses back. It's super efficient in terms of your time.

There are pros and cons to doing moderated versus unmoderated. With unmoderated, you can't change the script. So it's a little bit less flexible that way. But 90% of the time, I'll do unmoderated. If you're talking to a UX person, they probably have really strong feelings about it and be able to go into detail about when you should use one or the other.

But for me, the only time I'm using moderating testing is if I'm targeting a pretty unique audience that I can't reach on one of the self-serve platforms. For example, when I was working with Jobber -- a company that makes software for home services people like plumbers and electricians -- I couldn't find any of them on usertesting.com. So I actually manually recruited some people and did it over Zoom, but it's definitely harder that way. So I prefer the self-serve options.

They are super convenient. It's usually around like 50 bucks per session, and the sessions are usually about 15, 20 minutes. Most of the time, the quality's quite good. There have been some cases where I just didn't get anything useful from the session or the person didn't follow the instructions well, and in those cases you can often get a refund, but most of the time it works really well.

When you're using these tools, typically people know they need to speak aloud as they're answering these questions, and I think the tools will even have a reminder to that effect. You can include that in the task itself if you want to be extra clear.

I'm mainly working with B2B SaaS companies. I'd say about 80% of the time when an onboarding audit, questions and tasks are gonna be pretty similar. So it's... go to the homepage, evaluate the homepage. Is it compelling? Go to the pricing page. Do you understand the pricing? Then go through and sign up and so on, until ultimately they're looking at the upgrade page. Then, answering whether they have the information they would need to upgrade and whether they would be convinced to upgrade at this point.

I'll also include some specific questions -- normally around things I might have noticed in the UX walkthrough or things that the client might have asked me to focus on. So if there's a hypothesis that people can't easily find the upgrade page in the product we might ask a specific question around that. Or if there's a hypothesis that it's not that clear where you go to invite teammates in the product, we could ask a question about that, too.

Mike Fiorillo

When working with the team at Clubhouse (now Shortcut), I used this usability testing script to better understand the user onboarding experience.

Have 5-10 users go through the process of signing up and using the product for the first time.

These should be target users, ideally with no prior experience using the product.

Spend 1-2 minutes looking over the homepage page. Is anything unclear?

Go to the pricing page. Is it clear how the pricing works?

Go through the account setup flow, and describe what you like or dislike about the experience.

What’s the first thing you would do after setting up your account?

Do you understand the relationship between stories, epics, milestones, etc?

How would you go about inviting teammates to Clubhouse?

Imagine you were ready to upgrade your Clubhouse account. How would you do so?

Onboarding usability testing script for Clubhouse

Script plan & prompts

Plan:

Prompts:

Sign up to see more