Throwing Spaghetti on the Wall…

success-2081168

Have you ever heard someone say: “Don’t just throw spaghetti on the wall and see what sticks.”?

Well, obviously that’s not a good strategy to understand priorities and inform a future course of action. It’s also messy and a little disgusting…

A much better approach is to understand problems and drill down to root causes, identifying cause and effect correlations, and then formulating a set of hypotheses on how to influence those root causes. But let’s start from the beginning…

The spaghetti approach

Here is how I know when a PM interview doesn’t go well:

Me: “Interesting problem. How did you find out how to solve it?
Aspiring PM: “I did A/B testing and looked at the results.
Me: “Sounds cool, how did you know what to test?
Aspiring PM: “Well, we tried out a bunch of things, and then picked the one that showed the best results.

That’s not experimentation, at least not in a scientific sense, that’s classic throwing spaghetti on the wall and seeing what sticks. It’s expensive. With this method, you will find the right solution only by brute force or sheer luck. More often than not, the true solution and needle mover will remain elusive.

If you want to get to the true best global solution through experimentation, you need to have a plan first!

Drop any preconceived notions of ‘the right solutions’. In fact, burn your list. Instead, start from identifying the root causes and focus your experiments on understanding what drives those root causes.

The scientific Method

Experimentation is like throwing pebbles. If you have a plan where to throw them, you will likely hit your targets with a few throws. If you don’t, you will need a LOT of pebbles to hit anything worthwhile.

Here’s how you develop a plan before you start throwing your precious stack of pebbles:

Step 1: Root causes – What is the problem?

Start with identifying the problem. Then ask yourself what causes that problem. List all the drivers that you can identify from the data and observations that you have available.

Check for causations. Are those drivers really causing the root problem, or are they just correlated? Drill all the way down until, based on the data you have available, you cannot draw clear cause-effect relationships anymore.

Step 2: Hypothesis – Enter the unknown!

Up to here, causations were directly supported by existing data and observations. Now they are no longer, and you need to find ways to fill your data and knowledge gaps. You start making a plan for throwing your pebbles.

Start to develop hypotheses for the cause-effect relationships for which you don’t have clear data. Check if there are any drivers that you might have missed. Where do you have hunches (informed guesses), but no data?

Step 3: Experimentation – Closing the data gaps.

You have several brilliant but untested hypotheses. Now is the time to come up with a plan to put those hypotheses to the test. It’s time to develop experiments that can validate your hypotheses and provide you with the missing data.

Be clear as to what data specifically you need to get from an experiment to validate your root cause hypothesis. You can get a lot of data out of experiments, but not all of it will truly correlate to the specific needle that you want to move.

Think creatively and broadly as you get into designing your experiment. Not every experiment needs to be a big engineering project.

There are many ways to get data. Experiments can be product implementations, but they can also be very simple initial and manual tests with small groups of users or user research studies. Of course, the closer your experiment is to a large scale production roll-out, the more precise your data will be. However, you don’t always need that precision for the initial validation of an idea that will inform the next steps in a project.

The faster you can get results, the better. Sometimes you need to build something out in scale to get the right data; more often, you don’t. There are no bonus points for expensive and slow tests.

Step 4: Refinement – What have you learned?

Look at the data. See what hypotheses are validated and which ones are not.

Don’t leave it with that simple checklist though. Reflect on how your cause-effect framework might have changed with the new data and insights. Does the experiment’s data indicate new root causes that you were previously unaware of?

Finally, ask yourself if you have answered enough of your root cause questions to build your MVP, or if you need more experimentation and data to ensure you will head out in the right direction.

Sticking points

  • Experimentation is great!
  • More specifically, targeted experimentation is invaluable to get missing data and understand your space.
  • Just trying out stuff, on the other hand, is wasteful and will likely increase confusion instead of reducing it!

 


Did you like this article? Want to read more?

I will keep posting articles here and I have them lined up way into summer 2020. However if you want to get it all in one comprehensive, structured, and grammar-checked (!) view, check out our new book:

 

Put on your oxygen mask first - book cover

Put On Your Own Oxygen Mask First

A practical guide to living healthier, happier and more successful in 52 weekly steps

By Alfons and Ulrike Staerk

ISBN 9781077278929

Find it on Amazon: Paperback, Kindle

 

If you like what you’re reading, please consider leaving a review on Amazon. If you don’t like it, please tell us what we can do better the next time. As self-published authors we don’t have the marketing power of big publishing houses. We rely on word of mouth endorsements through reader reviews.

Innovation! – And how to pitch it

Edison

Innovation

We are all excited about seeing innovation, and even more so, driving innovation. But how is innovation happening? More interestingly, how can we drive innovation?

Modern Entrepreneurs will tell you that it takes many attempts, and often countless experiments and pivots, to find an idea that sticks and takes off. That wisdom is not new though. As a kid, I saw a movie about Thomas Edison, and I never forgot this quote:

“Genius is one percent inspiration and ninety-nine percent perspiration.” – Thomas Edison

Edison tried hundreds, maybe thousands of different materials, until he finally got his first working lightbulb.

The same is true for innovations that we want to drive. We need to think through many ideas, vet the ones that make the most sense to us, and develop them further. Then we need to pitch them to see if they hold water under scrutiny. Finally, we must test if the assumptions hold true in the real world, or are just paper dreams.

We will pitch a lot, and only a few of our ideas will see the light of day. But that’s ok, it’s actually good and intentional. Unless you are a once-in-a-century genius, just like it did for Edison, it will take you several attempts before you come with an idea that gets broad support and quick traction. That’s good, the selection of the best ideas is a critical part of the process. In our case, Amazon is successful because it focuses on the most promising ideas, not everything we could tackle (otherwise we would still be selling books from a garage).

Pitching

So how do you pitch an idea?

Of course, there are events through the year, that proactively solicit ideas – unconstrained ones like Hackathons and Ideathons, or more focused ones like 3 Year Plan and roadmap planning.

However, I hope that your ideas are not limited to those fixed and defined dates, but your creative juices flow every day at work. So how do you land those sparks of inspiration?

First of all, please don’t wait for the next ‘official’ planning session! Second, don’t trust the ‘elevator pitch’. It’s highly unlikely that a thoughtful and responsible leader will invest substantial resources based on the information and depth they can get from a quick hallway chat or an improvised meeting.

Instead, the mechanism we have at Amazon to pitch ideas and sell them is through PR FAQs. PR FAQs and Working Backwards docs are Amazon’s way to explain a proposal, think through the critical details, and win approval to move forward, experiment and learn more along the way.

Here’s a quick overview as to what makes a good PR FAQ (at least in my mind):

Be clear, be focused – You need to focus on the core problem that you want to solve, the core proposal you want to make. Don’t provide a laundry list of stuff, make a pitch for a core idea (you can still move your laundry list to the end of the appendix if you have separation anxiety). You want your PR FAQ to be a clear, crisp and concise story. You also want it to be a believable pitch so that your leaders can trust that you have thought it through.

Understand your ‘What’ – What problem are you solving or what new value are you creating? Specifically! Pick one, not ten.

Explain the ‘Why’ – Answer the “so what” question that every leader has. Why should I care? Why does this matter? This is where you MUST have data – our beloved Xs and Ys just don’t cut it here. As you explain the “why this matters”, also think about the “why now”. It’s a great idea for sure because you are a smart person, but of all the things, why should we focus on this right now, and not in a year or two?

Be diligent about the ‘Why not’ – Pitch your idea, but don’t be a sales rep (sell it, but sell it honestly). Tell the full story, not the rosy picture. Don’t drop valid concerns to drive an easier sell. What are the risks, challenges, and headwinds? If you proactively address them, and explain how you think about testing or mitigating those risks, your credibility will jump to the next level.

Lay out the ‘How’ – Let’s assume, we buy your idea, now what? How would we execute? How can we test your assumptions (see the next point)? What are the next steps that we need to take if we want to go further?

Clarify the ‘Asks’ – What do you need? Are you asking for time or permission to run an experiment on your own? Are you proposing to rebalance current funding? Are you asking for incremental funding? Be clear, not ambiguous! What’s the goal of the doc? What do you want to achieve?

At this stage, keep in mind, that it takes many attempts and many shots to develop, pitch, and land a new and compelling idea. Some will land, others won’t.

Early validation

One of the first questions you answer when you think about your proposal and draft your PR FAQ should be: How do you validate that you are onto something? How can you get early signals, as to whether this will actually work in real life?

You must be explicit and specific in the experiment you will do as a next step and litmus test. That experiment needs to be something simple. It can be a proxy for what you actually want to build. It can be something that you run manually, and with only paper support, before we all take the next step and decide to invest in engineering. It can be something that you do in shadow mode in parallel before we decide to change process or customer experience. For example, you can just show a few paper mocks to future users and see their reactions. Whatever it is, you need to make it a point to find ways to test quickly and cheaply.

You need to break big and scary proposals down into smaller bits that are less expensive to fail and learn, and then quickly test them before you move on. If you don’t do that, you are asking for a big leap of faith from your leaders to make big investments in a paper-idea, that very possibly could not work after all.

We all want to experiment, fail (we don’t want to, but we are aware that it’s part of the process), and learn fast. Make it a core part of the proposals you are putting forward. In every single proposal, think a LOT about how you can experiment cheaply in the early stages, to provide quick validation or signals to pivot.

 


Did you like this article? Want to read more?

I will keep posting articles here and I have them lined up way into summer 2020. However if you want to get it all in one comprehensive, structured, and grammar-checked (!) view, check out our new book:

Put on your oxygen mask first - book cover

Put On Your Own Oxygen Mask First
A practical guide to living healthier, happier and more successful in 52 weekly steps
By Alfons and Ulrike Staerk
ISBN 9781077278929

Find it on Amazon: PaperbackKindle

If you like what you’re reading, please consider leaving a review on Amazon. If you don’t like it, please tell us what we can do better the next time. As self-published authors we don’t have the marketing power of big publishing houses. We rely on word of mouth endorsements through reader reviews.