Validation testing makes your business plan bulletproof. Here’s how…
When you think you’ve got a good business idea and it sounds great in presentations or works well on paper, you probably want to get it going as fast as possible. But executing your idea prematurely, especially if it involves a large investment into a supply chain or software, is never a good thing.
Validation experiments help you confirm that your idea is more than a hallucination and that it’ll take off when you launch it. They’re an opportunity to expose your idea to the practicalities of the real world to see if it will hold up.
In the validation process, you’ll conduct rapid experiments so you can learn and adapt to ensure real-world desirability, feasibility and viability. The more experiments you do, the more insights you’ll gain, and the more bulletproof your business plan will become.
Source: crowdfavorite.com
But for startups, testing can use up a large chunk of the limited funds you’ve got available. These are the cheapest ways you can validate your ideas to save face in the long run.
3 Validation Experiments to Try in 2022
Before we get into the experiments, you need to come up with a hypothesis that your value proposition can build on, and should outline the things you need to learn from your validation experiments. It should also be testable, precise, and discrete.
Testable: Can be proven with evidence as true (validated) or false (invalidated).
Precise: You know exactly what success looks like for this experiment.
Discreet: Describes only one testable and precise thing you want to investigate.
Once you’ve got your hypothesis, you can start selecting experiments that will prove or disprove it. According to Strategyzer, to pick the right experiment you need to ask these 3 questions:
- Type of hypothesis: What type of hypothesis are you testing - will it answer questions of desirability, feasibility, or viability?
- Level of uncertainty: How much evidence do you already have (for a specific hypothesis)?
- Urgency: How much time do you have until the next major decision point or until you run out of money?
Remember, the less evidence you have, the less certainty you have. Quicker, cheaper experiments are better if you’ve got very little evidence.
The 3 experiments we’re about to share with you are all cheap, but we’ll also look at how long they take to set up, how long they should stay set up for, and the strength of the evidence you’ll gain from them.
They work best when done one after the other. If you get poor results, you should pivot your strategy and go back to experiment one. If the results of all three indicate that your business idea is worth pursuing, it’s time to take action.
1. Simple Landing Page
This experiment tests if your value proposition resonates with your customer segment. Your simple landing page must do two things: outline your value proposition, and call your audience to action. The rate of response to your call to action will give you an idea of how effective your value proposition is.
To do this experiment, you’ll need to build a landing page with your value proposition and a call to action (such as an email signup form) above the fold. Below that, you should include some text that positions your product or service as the solution to your customers’ pains. This copy should lead with the benefits or gains that your business promises. Here’s a great landing page example:
Once you’ve built and launched your landing page, you can start driving traffic to it. These are some methods you can use to drive traffic:
- Online ads
- Social media campaigns
- Email campaigns
- Redirecting existing traffic
- Word of mouth
- Discussion forums
Keep track of the analytics throughout the experiment’s run time. How many people visited your page? How did they find your page? How long did they spend on it? Did they respond to your call to action?
You can A/B test different versions of your landing page to see which variation of your value proposition has a higher conversion rate. We suggest tools like Google Optimize, Freshmarketer, and Crazy Egg to A/B test without writing all the code yourself.
This is one of the most affordable ways to test your ideas at scale. Here are a few other things to consider:
Setup time
It shouldn’t take more than a few days to set up this experiment. There are plenty of drag-and-drop or no-code/low-code website building tools available to make this step as simple and quick as possible. However, crafting your value proposition may take significant time.
Run time
Your run time will be dependent on how effectively you can drive traffic to your landing page. Your goal should be to have 100 unique visitors per day for a few weeks to gain sufficient evidence. If your daily traffic is lower than that, you’ll have to run your experiment for a longer period.
Evidence
This experiment is all about calculating the conversion rate of your value proposition. To work this out, you’ll need to track the number of unique views and the number of responses to your call to action. So if you’re calling viewers to sign up to an email list, the number of signups is what you’re counting.
Unique views / Actions = Conversion rate
Eg. 1000 views / 250 email signups = 25% conversion rate
You should aim for a conversion rate of 10%-15% for a new startup idea. This is above the average of 2%-5%, and will show you that your idea is worth moving forward with.
Once you have a landing page up, you may want to make tweaks or conduct mini-experiments to improve the conversion rate. Make sure to space them out consistently and stick to one change at a time, as suggested in the book, The Lean Startup. You should aim to build one feature at a time, measure the results of that change, and learn whether it worked and what the next best change could be.
Keep hold of all your data. You can look at your conversion rate over time and see which mini-experiments paid off the most.
Your measured action doesn’t have to be email signups. You can also call your site visitors to buy a pilot of your product for a premium price, there and then. This will give you very strong evidence that shows whether people are willing to put money towards your product. If they are, then you’ve got evidence that the problem you’re solving is urgent and your chances of success are far higher.
Skills
To set up a simple landing page you’ll need a designer and a copywriter who can clearly communicate your value proposition both visually and with words. If that’s beyond your skill set or budget, here are a few cheaper (or free) alternatives:
- Images: Unsplash or Pexels
- Copy: copy.ai
- Drag and drop website builders with templates: Wix or Webflow
If you want something more custom, you can hire freelancers from Upwork or Fiverr to create your landing page for you.
Once you’ve completed this experiment, you can interview the site visitors who signed up to your email list to find out what made them sign up.
2. Wizard of Oz
This cheap validation experiment yields extremely strong evidence, but can only be done on a smaller scale. You and your team work behind a magic curtain to manually deliver the service that your product will deliver when built. This means that you can test out your business idea without actually having to build anything.
Source: AnswerLab
In most cases, the magic curtain will be your landing page and customers shouldn’t be aware that the process behind it is manual rather than automated.
Wizard of Oz is an effective way to work out the right time to scale your business. If you scale too soon, you risk failure and if you don’t scale at all, well, you won’t scale at all. With this experiment, you’re looking to answer the following questions from Strategyzer:
- How many customer requests could you perform manually each day?
- What is the cost to deliver each one (cost structure)?
- What is the most customers will pay (revenue streams)?
- At what volume is it more cost-effective to automate these tasks?
From here, you can set a specific threshold for when it’s time to automate the business.
If this is sounding a bit complicated, here’s an example of how we do Wizard of Oz experiments at The Delta:
The cost of this experiment depends on what it costs you to supply your product. The magic is that you don’t have to put money into automation, just the manual delivery of your product. These are some other variables:
Setup time
Because of the manual nature of a Wizard of Oz experiment, setup time can be on the longer side. If you’ve already built a simple landing page you can continue to use it as your ‘curtain’ where customers can request your product or services. Simply adjust the call to action accordingly.
Then you’ll need to map out the steps necessary to deliver the customer’s request and create a manual system using software like Google Sheets or Airtable for tracking orders and steps. It’s also wise to test your system with a colleague before proceeding.
Run time
The run time depends on your ability to drive traffic and sales as well as the specifics of your product or service. Before you start, set a goal of how many customer requests you’d like to receive during this experiment. Depending on your marketing, it can take you days or weeks to reach this goal.
Evidence
The amount of time that goes into setting up and running this experiment is worth the very strong evidence you’ll get back. The repeated task of manually fulfilling customer requests will give a clear indication of how long it takes to satisfy customers manually.
You can also measure the conversion rate of your idea and how much people are willing to pay for it. Willingness to pay is very strong evidence of the desirability of your product or service.
You’ll also be able to gauge customer satisfaction. The evidence here is particularly strong because you’re asking for feedback on a real experience rather than a hypothetical business idea.
Skills
To complete a Wizard of Oz experiment you’ll need all of the necessary skills to manually create your product or service. You’ll also need to have a landing page to act as your ‘curtain’, and you’ll need to be able to drive traffic to it and convert that traffic to sales - so some marketing skills would be helpful.
With the Wizard of Oz experiment, you’ll gain a pool of customers who can give you invaluable feedback that will further shape your business idea.
The next validation experiment is all about checking in with that pool of customers to gain invaluable insights that you can’t move forward without.
3. Validation Survey
Validation surveys collect information from existing customers about a specific topic. The main things you’ll want to find out from these surveys are:
- How disappointed customers would be if your product went away
- How likely customers would be to refer people to your business
This extremely affordable experiment does require existing customers that you’re able to contact either online or via email. The customers gained in the Wizard of Oz experiment above will be sufficient, or this can be done as soon as you’ve launched your MVP and have picked up a few customers.
There are two tests worth conducting to gain the best evidence for this experiment: the Sean Ellis Test, and the Net Promotor Score (NPS).
The Sean Ellis Test, named after its inventor, involves measuring desirability through scarcity. Ask customers, who have used your product more than once in the last two weeks, if they would be disappointed if your product went away. If they wouldn’t be disappointed, then your product isn’t desirable enough and scaling it would be a huge financial risk.
The Net Promoter Score survey should take place after the Sean Ellis Test for best results. It asks customers to indicate on a scale from 0 (not at all) to 10 (extremely likely) “How likely is it that you would recommend this product to a friend or colleague?”. Scores from 0-6 are considered detractors and scores of 9 and 10 are considered promoters.
The NPS is worked out with this formula:
Source: trustmary.com
Other types of validation surveys include brand awareness, customer satisfaction, and customer effort score (ease of use) surveys. Here are more details about this method:
Setup time
The setup time is minimal, as a questionnaire can be put together in a matter of hours or a day at most.
Run time
If your audience is responsive and you’ve got good channels of communication with them, you could have sufficient data in less than a week. This is where having a responsive email list is beneficial. If you have a harder time contacting your audience and compelling them to fill out your questionnaire, this task could become drawn out.
Evidence
While this experiment isn’t the strongest in terms of quality of evidence, the data you gain is well worth the minimal amount of cash you’ll need to put in.
- How disappointed would you be if this product went away? This question gives you an idea of what your churn rate might be if you were to scale your business. If people wouldn’t be disappointed, the chances of them continuing to be loyal customers is lower. At least 40% is sufficient here, i.e. 40% of your customers would be disappointed if the product disappeared.
- How likely would you be to refer this product to others? Anything above 0% on the NPS is good for this experiment, but you should check what the benchmark would be for your industry.
Skills
You’ll need to have marketing and research skills to get this experiment right. Questions need to be crafted extremely specifically to yield accurate data, and customers need to be segmented to reduce noise. For the survey itself, you could consider tools like Typeform, SurveyMonkey, and the forms feature in Airtable, if you’re already using it.
Time to take action
The simple landing page helps to show that your idea is desirable, as people would engage with your CTA. The Wizard of Oz helps to show it is feasible, as you can actually provide the value/service. Finally, the validation survey can help show your idea is viable, as you can retain enough customers to grow the business sustainably.
These 3 experiments work best when done in the above order. If one of these experiments reveals flaws in your idea, you’ll need to pivot your business plan and go back to experiment one — the simple landing page.
However, if the results of one experiment show that your idea is desirable, feasible, or valuable, you can move on to the next one with more certainty that your idea is worth putting more time and money into.
Here are a few rules of thumb to keep in mind:
- Keep things cheap & fast in the beginning
- Conduct multiple experiments for the same hypothesis to ensure strong evidence
- Reduce uncertainty as much as possible before you put money into building anything
- Pick experiments that produce the strongest evidence within your financial and time constraints
Keep validating your idea until all of the evidence says it’ll work. Then it’ll be time to take action.
If you try any of these validation experiments or you’ve got any burning questions, start a conversation with us on LinkedIn.
Header image source: Strategyzer.com
