The value of experimentation in innovation
Let’s say you’re a product manager, innovation coach, team leader, or even a start-up founder. And you’ve come up with an amazing new product or service. So, you assemble a motivated team to pitch this raw idea to them.
They love it.
With everyone on the same boat, your team gathers around the drawing table to make things tangible. You will, of course, need resources and a project designed to bring your idea to life. So you plan a thorough presentation for stakeholders and investors.
Guess what? They love it too.
Things are looking up. With the resources, and a one-year deadline, this project is bound to be an idea no-more. As time goes by, you see how ideas take shape.. The first version of your product or service has been born, and it is just as you all imagined.
Your team loves it.
Your team puts their efforts into a full-fledged online and in-store campaign to launch your product. After a long time of work and commitment, your idea is ready to be put in front of consumers.
Business seems a bit slow. The weather might’ve influenced the sales.
Budget expansion. We should’ve spent more resources on our marketing.
We have 30 conversions and subscribers. What do these numbers mean?
After some tough meetings, you’re forced to pull the plug. It’s simply not what your stakeholders and investors expected.
The fundamental truth
Our tale is a simplistic version of reality that reflects a fundamental truth that most people, who wish to turn an idea into a market-breaking novelty, face.
Project teams, innovation incubators and programs, and companies get inspirational ideas all the time. Falling in love with the idea is usually what follows. The end-result being hefty investments and a significant amount of resources in its development.
The reason is simple – people build things they like. And if they like it, consumers are bound to like them as much, right? Unfortunately, following hunches without validation usually turns into irrelevant products on the market.
The one thing that can prevent wasting valuable resources is early contact with actual consumers. One of the core ways to get their input is by experimenting with different value propositions, designs, and messaging, to mention a few.
Spotting the problem
However short and un-poetic, our story is meant to shine some light on what seems to be the universal challenge: what causes product launches to fail. And it all leads to lack of evidence and relevant data. These are essential to support a specific product or feature, but they will also delimit your future decisions on how to tackle product development.
Big or small, companies repeatedly make choices based on biased thoughts or gut feelings. But regardless of their experience in the field, thinking something will work isn’t enough. Without experimentation, results are sub-optimal at best.
A practical approach; the benefits
Because experimentation can’t happen without proof; whether you’re offering a new product or service, you’ll have to look at the scientific method of experimentation first. This process is supposed to move fast and iteratively, and will leverage data-driven and evidence-based decisions.
Tests offer more than information that influences your next set of decisions, they add value to the entire product development process. And here’s how they do it:
Product experimentation works, in a way, like human behavioral testing. Testing if something works beyond thoughts and ideas translates into putting your product in contact with real end-users and consumers.
Gathering data and evidence about your biggest business assumptions isn’t only about finding hard-data that supports your experiments. By doing so, you’re also minimizing commercial risk, making sure you won’t be investing into a product your users won’t want.
You can validate your problem and solution’s market fit without over-investing or risking. For each stage of the business design process, you can conduct specific experiments to generate a particular kind of proof.
This is how experimentation ends up taking advantage of the power of statistical significance, by extrapolating the learnings onto larger crowds. In other words, you don’t need a lot of time and money to get your experiment started.
Believing isn’t enough. Gut feelings will always be trampled by evidence-based outcomes. Experimentation lets you step away from guesses and opinions, and replaces them with facts and customer-driven data.
Is there such a thing as unreal intent? As a matter of fact, there is, and it’s related to stated behavior.
Digital experimentation allows you to measure demand for a new product by actually introducing it into the market. In contrast to more traditional surveys or consumer interviews, this gets you insights not only of what people say, but what they’re willing to buy in reality.
How do I get started?
1. Identify your pain points
The first step is the most challenging one. This is where you’ll identify the riskiest assumptions, and start designing and experimenting based on them. This part is particularly difficult as it requires distilling product ideas and turning them into singular, simpler assumptions.
Once you’ve gone through the first stage of the process, your risk assumptions are ready to be turned into hypotheses that will undergo experimentation.
Check out our Assumption Mapper to get started!
2. Design your experiment
With a hypothesis in place, you can design an experiment that will have a clear format, outlined channel, success and failure metrics, and a timeline to follow. The results will lead the way towards the ones worth pursuing.
We know that pinpointing the elements that should be a part of your experiment can turn into a maze. This is why we’ve designed an Experiment Card to help you capture the right ones, and start designing your own experiment right away.
3. Run your experiment
Once your experiment is live, it’s time to monitor your progress. The best part is you won’t have to wait long. Depending on the experiment you’re conducting, results should be ready to be analyzed in a matter of days or weeks.
4. Analyze and iterate
The main outcome of running an experiment will be a bunch of information, now it’s your job to do some data crunching with it. By diving deep into qualitative and quantitative figures, the information turns into evidence that will prove or debunk your hypothesis, but it doesn’t end there.
There’s plenty of additional learnings and insights to look forward to when conducting these tests. The validation results, and the bonus data will both directly depend on how well you designed your experiment.
5. Make evidence-based decisions
After studying the results of your experiments with your riskiest assumptions in mind, you and your team have all the information you require to make evidence-based decisions. You’ll be heading into the marketplace knowing if your consumers are actually fans of beef-flavor ice cream, Tinder for farmers, or if it was simply an idea that would only work in the heat of the moment
Let us not forget...
Experimentation loses value when it’s not an iterative process. Based on inputs and initial consumer results, this process is meant to be a loop, and even scaled with parallel testing. The more relevant information you gather, the faster you can move into decision making phases, and the more unexpected and validated ideas you’ll find along the way.
Curious about the types of experiments you can run? You can find some inspiration in our Experiment Picker tool.
