We would use this document when a new PM was onboarding onto our team or when someone new was joining product review meetings and wanted to learn how they worked.

This also helped us kick off how to use them generally as they weren't used consistently prior to me joining the company.

This is really important. As an exec, when you're in the room, you should state opinions as that -- opinions or an instinct. You need to frame it as a suggestion, not a directive. And you need to reinforce that the decision is ultimately the PM's call.

When you end up being wrong, you should acknowledge the debate, that you had been wrong, and highlight what you learned, especially when there is data that contradicts what you had thought before.

This also models behavior for when the PM makes a mistake or learns something.

Tom Willerer

At Opendoor, I used this product review process overview to help the team converge on a more consistent product review approach and to guide new PMs onboarding onto the team.

At Opendoor, we want Product Review meetings to focus on fostering debate, curiosity, and rigorous product thinking. They are a tool for driving innovation, strategic alignment, and debating trade offs. 

They do not have to be a decision making meeting to be effective, and – in fact –  often it’s better to not try to make decisions in a product review

. The mindset you should have going into a Product Review is one of high conviction but willing to be convinced that you are wrong or at least partially off course. You’d rather learn that now than after you spend the time and effort to launch whatever it is you are planning.

Product Review meetings should focus on the following dimensions. These can serve as a general outline for the pre-read you send out and for the meeting time together.

. We as PMs spend too little time on the problem space and  instead tend to focus on solutions. Solutions are more fun but as Einstein supposedly said, “If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and five minutes thinking about solutions.” The same should be true of Product Reviews. We should focus them on the key customer problems to be solved and drive debate and alignment around that.

 Once we’ve defined the problem space, it is important to then explore the range of hypotheses for why a particular solution might solve the problems you’ve identified (or Jobs to be done). Hypotheses are typically if-then statements, e.g. if we build xyz then qrs will happen and should be mapped to the core problems already identified.

The strategic alignment and differentiation.

 The PM needs to show why this particular problem/solution is worth spending resources on now given all the other competing priorities and how solving this problem will further differentiate the company and product from competitors, if at all.

The phase of the project you are talking about

. There are multiple phases to a project from early to late stage and can be thought of on a spectrum from the “Understand/Define stage” to the “Explore stage” and finally the “Refine/Ship stage”. Each of these stages require different feedback and from different people.

 It is critical to explain - up front - how you will know if the project is successful, and importantly, what is planned next given a few different possible outcomes.

Creating a culture where debate, curiosity and rigorous product thinking are valued is not easy. Far too often Product Reviews can either be too intense and competitive or too soft, fuzzy, and non confrontational. Product Reviews are not a place to “earn points” by making the presenter look stupid but rather should be seen as a team sport where we’re all helping each other succeed so that we as a team can succeed. Here are some tips for how to engage in Product Reviews to get the most out of them:

I want to set the tone in the meetings that the PM is the decision maker, not me

 My job is to provide input, question things, ensure the team is solving the right problem and most importantly to empower my PM.

As a cross-functional partner, your job is to question the logic, debate the validity of the hypotheses

 and whether the right jobs to be done have been identified and most importantly to clarify how this impacts your role, OKRs and the scope of your job going forward. Importantly, you should not present your opinions as a blocker (e.g. “I wouldn’t do that”) but instead open up new possibilities with phrases like, “How might we think about this more …” or “Have you considered xyz?”. Your job is to help the team presenting, so they need to hear you and not feel threatened by you.

As a core team member, your job is to help clarify when questions come up

, take notes and support your PM partner in exploring the topic at hand. If you hear someone with a dissenting opinion, dig in to learn more. Don’t shy away from dissenters, they are a gold-mine of new insights for you and might show you a blindspot.

As a PM, your job is to facilitate the meeting and effectively drive the discussion

. You are the ultimate decision maker with regards to which parts of the feedback you should allow to change your approach vs. which parts will not. The most effective way to do this is to ask questions, especially challenging questions, and focus on making it safe for people to deliver you the right feedback by:

Stating your recommendation clearly, and articulating why you are making the recommendation.

Ask a few open-ended (tough) questions to make it clear you are open to people challenging your assumptions.

Stating the areas where you are most interested in getting feedback, and guiding the discussion back to these points frequently.

After the meeting, you should synthesize what you heard and clarify what next steps are coming out of the discussion.

Product Review meetings should happen at least 2x per month.  They should happen prior to initiatives launching or being tested or after a team has initial results/learnings and is deciding on what to do next (e.g. scale, continue to iterate, etc.). In addition, Product Reviews can be used during planning (either quarterly or yearly) as a way to vet the higher level product strategy that is being set forth by each team. It is best to have a set day and set time for Product Reviews, and Thursdays tend to be a great day because it allows for prep time earlier in the week and a day to debrief on Friday.

The leader of the meeting will be the PM running the project. They are expected to determine whom from their team should be in the meeting. The general guidance is to keep meeting attendance to the folks that are critical to the meeting or project success (e.g. no more than 10 people and all should be active participants).

Required: Key Eng, Product and Design leaders

Team to determine the rest of the attendees, which will vary per meeting.

 Some topics might require slides (e.g. if a more UI based product review), so don’t feel restricted by memos, but to the extent possible writing down your thoughts should help clarify them. These meetings should ideally save the first 15 minutes for 

 and/or any presentation from the PM and 45 minutes of discussion.

Why is it better to not make decisions in a Product Review?

Product review meetings are often slightly larger than decision making meetings, as there is a part of the product review meeting that is about sharing context and allowing a broader group to pick up on the cultural elements for how your company does product development; whereas, decision making meetings are smaller, more focused meetings with only the necessary stakeholders in them.

Decision making meetings tend to be about presenting to and hearing from an executive or empowered decision maker; whereas product review meetings are about fostering debate, curiosity and rigorous thinking. This type of thinking can sometimes be stifled when an exec is really digging in to make a decision, thus it can dampen the type of product development culture that can come from doing product reviews correctly.

How can I assess if my Product Reviews are effective?

The best way to determine if your Product Review is successful is if you, the PM, walk away from the meeting with actionable feedback on the areas you identified as needing feedback. Additionally, you can informally gauge whether the participants in the meeting felt heard. If not, you might have too many people in the meeting or the material wasn’t structured well or the meeting went poorly.

Should Product Reviews change for 0-1 work vs. feature development?

The questions are the same regardless of whether the work is 0-1 or feature development. You should be able to talk deeply about the problem you are solving, your hypotheses, the strategic importance, the phase of your development, and the key success metrics. One difference though is that 0-1 development might be on a faster pace and less formal, so every day might feel like a product review but using this structure can be helpful regardless of the type of project.

What do I do if my Product Review is going poorly?

One way in which a Product Review goes poorly is when too much feedback is being delivered in an unstructured way, both in a doc and live in the meeting. The way to combat this is by making it clear in the memo you write what you want feedback on, and coming back to this section as a guide to the meeting. It might also be a good idea to create an agenda with time constraints on each area where you want feedback to provide structure to the meeting. Another failure mode is if there is no feedback delivered in the doc or the meeting. This can either mean people lack sufficient context in order to engage, in which case you likely need to give them the right context or include different people, or everyone agrees with you. If everyone agrees, maybe the topic isn’t right for a product review or you need to do a better job of surfacing the tradeoffs and areas of potential disagreement that might come up to foster the conversation.

Product review process at Opendoor

How We Do Product Reviews At Opendoor

Approach

Focus

Sign up to see more