Experiment Launch Checklist from Alexey Komissarouk
Experiment Launch Checklist
This is obvious, but: Before launching an experiment in production, it’s well worth running through a checklist to avoid common classes of failure. Here is a sample checklist (feel free to adjust as necessary)
- Before Launching an Experiment into Production (often before merging the PR)
- By somebody with context on the experiment but not the engineer who wrote the code (due to blind spots). The Product Manager is an ideal person to run through, but Design, QA, or an Engineering Manager are other good options.
Part 1: Tablestakes
- Is there an actual, well-defined set of test steps (example)?
If not, have the PM or Engineering Lead write these up