This document gives an understanding of how I approached to validate the hypothesis during the early stage of product life cycle. The frameworks & techniques helped the team to consistently learn about the problem space & eventually to find the right value prop for our users.

Experimentation helped us to gather evidence in areas with higher levels of uncertainties with possibilities of higher impacts. We adopted two methodologies to experiment: based on the risks involved we adopted building a ‘minimum viable feature’ (high evidence) and A/B Testing (low evidence) to validate our hypothesis.

Experiment has its own risk because we are changing something which is already working and we want to improve it. Business teams need to be informed where will be the highest impact for the business (whether success or failure).

So we went with defining the problem statement, impact, risk. Lastly we felt our approach or tools we use to do the test may also impact the results. So, we captured the approach such that you can change the approach to improve the results in the next iteration.

Many in our org have been inspired by this story telling and starting using this as a reference.

The results showed that Variant had a better conversion rate. We could see that user curiosity is increased to check the special pricing. At the same time, rate of improvement is not magical which made us to learn that customers are still looking for a better product - more than just a special price.


Ramani Siva Prakash

This frameworks was created to help me summarize and communicate the outcomes of experiments while also providing some background info for stakeholders.

Experimentation Summary at Thoughtworks

Sign up to see more