HYPOTHESIS & EXPERIMENT (PART 2): Hypothesis, experiments and the 3 lenses of innovation

In the last post we established that when you and your company are trying to create a product, more often than not, problem and solutions get mixed up and that you your biases, company culture and other forces will try to pull you away from the problem space and into the solution space. Don’t allow that to happen!

In the journey to understand Hypothesis Driven Development, Lean Startup and related ideas I have found that both problem and solution need to be validated and that there is a large amount of tools and content to help (and sometimes confuse) you and your team to do that. It can be daunting and overwhelmingly frustrating to try learning this stuff, much else convince your team and stakeholders to apply some of these techniques.

Dan Olsen describes that the reason for creating his book, The Lean Product Playbook was to help understand how to apply Lean Startup concepts in a more tactical level.

Side note, I have read the book from cover to cover two times and I’m sure I’ll read at least some chapters again by the end of this semester. If you get a chance, grab his book or take a look at his youtube channel: https://www.youtube.com/channel/UChx1ibI5TI7bBva0Z21azrw. In fact, stop reading this right and go check it out. You can always come back later.

Anyway, before diving into frameworks and techniques (which we will be the focus point of my next posts) I thought I would talk a little about Assumptions, Hypothesis and Experiments.

Assumption vs Hypothesis

I’ll try to keep it simple: in my opinion assumption and hypothesis may be used interchangeably in the context of product validation. In my research I have found that the following are good definitions:

Assumption: something taken as being true or factual and used as a starting point for a course of action or reasoning

Hypothesis: an idea that is the starting point for making a case or conducting an investigation

These are Webster’s Dictionary definitions.

If you’re looking for a good post about assumption vs hypothesis, take a look at Kromatic’s view on the subject: https://medium.com/@Kromatic/assumption-vs-hypothesis-to-the-death-df1ebc63e749. In my opinion, this is the best content about these two ideas The examples he shares are very simple and clear. Here our two of them directly from this post:

In my experience, I learned that you usually start with an assumption before getting to a hypothesis. I think that it is the case because assumptions are simpler and they come in numbers; meaning that clients, stakeholders and team members will toss them around at interviews and brainstorming sessions.

If you want to further investigate hypothesis I recommend Teresa Torres’s series on hypothesis: https://www.producttalk.org/hypothesis-testing/. Particularly this one about the components of good hypothesis: https://www.producttalk.org/2014/11/the-5-components-of-a-good-hypothesis/

Also, take a look a this post about incorporating assumptions into Teresa Torres’s opportunity solution tree: https://www.producttalk.org/2019/12/adopting-opportunity-solution-trees/. Which is a tool I’ve been using for the passed three months.

From assumption to experiment; here is how I think of assumptions and how to convert them to hypothesis in order to validate by experimentation:

1. Gather all the assumptions about a idea (either a problem or a solution).
2. Narrow it down to the basic and most important assumptions for validation.
3. Transform an assumption into a hypothesis using the concepts cited above.
4. Choose the appropriate experiment for testing the hypothesis.

Try to involve the end user whenever the situation presents itself. Especially when it comes to desirability. Also, a great tool for mapping out assumptions in a collaborative way is an assumption map. Here’s a version from the miroverse: https://miro.com/miroverse/assumption-jam/

The Three Lenses Innovation

I’m getting ahead of myself here. Before going any further on the subject of mapping experiments to hypothesis we should talk about the 3 lenses of innovation: Desirability, Feasibility and Viability.

The good folks from IDEO have given us this mind model to think about our product, features and services and I think it represents very simply and gracefully what should be the focus any product’s continuous discovery activities.

Desirability should always be your initial focus. Alex Cowan calls it the independent variable and says in his course from Darden School at UVA that desirability is so important that it drives viability and feasibility and these don’t matter separated from desirability. I totally agree.

So, gathering what I’ve written so far, for each of these areas there are problems and solutions to be discovered. Also, you should start with desirability because, well as Metallica (the band) would put it, nothing else matters. =]. Well, they do matter, but not independent from desirability. This means you should look for problem assumptions and validate by creating hypothesis and experiments. Once you do that, once you validated your problem you should validate your solution.

Things I learned while trying to implement:

1. An experiment can be as simple as a couple of interviews with clients or potential clients. Or a benchmark or competitors’ analysis.
2. Sometimes you can test both problem and solution at the same time. For example testing A/B Value propositions by describing problem and solution to potential customers

Mapping experiments to IDEO’s Three lenses

Lastly, David J Bland says in his tweet that the following slide is his most popular slide, and for good reason in my opinion. I love it as well. He basically list many types of experiments that are design to test hypothesis related to each area of IDEO’s three lenses. Brilliant.

Conclusion

1. Problems and solutions should both be validated.
2. Start with assumptions, move to hypothesis and choose your experiment accordingly.
3. Segment your research and create hypothesis and experiments in relation to IDEO’s three lenses.