An innovation experiment is useful when it reduces uncertainty about a decision. A polished prototype or enthusiastic presentation does not establish demand, feasibility or sustainable economics. This article explains how to review a bounded experiment and decide whether to continue, change the approach or stop before committing more resources.
Start with the decision and hypothesis
An innovation experiment is a limited test that gathers evidence before wider commitment to an idea. It may use a prototype: a simplified version for trying one aspect of a solution. A hypothesis is a specific, testable expectation, not an established fact. Write the decision the experiment will inform. For example, should a service team offer a guided application review? Then state a testable hypothesis: a particular customer group can complete the required steps with fewer avoidable corrections when given a short guided review. Keep demand, usability and financial feasibility as separate questions.
The business innovation overview discusses the wider discipline. This review is narrower: it evaluates one uncertainty with evidence. ISO 56002:2019 provides guidance for innovation management; the practical review sequence here is an illustrative approach, not a mandatory checklist from the standard.
Agree the test conditions in advance
Define the participants, observation period, process, budget and evidence to collect. Specify what would support the hypothesis and what would justify changing or stopping the test. Choose thresholds for the local decision and explain their basis; do not present an arbitrary target as an industry benchmark.
Identify privacy, safety and service obligations before inviting participants. Use an approved process for any personal data and make material limitations clear. Where failure could cause harm, test in a controlled environment with qualified oversight. A small experiment does not remove the duty to protect participants.
Use an observation card for each attempt
Before the experiment, prepare a card for the task, conditions, result and assistance provided. A temporary identifier may be sufficient without recording the participant’s name. Record the observation date and prototype version. Changing a screen or its instructions during testing can change the user experience. Separate results before and after any change rather than pooling them without explanation.
For the guided application example, note whether the participant started independently, where they stopped, what they asked and whether they finished after assistance. “Application completed” does not show whether the design was easy to use or an employee effectively did the work. Keep brief descriptions of observed behaviour, with the team’s interpretation in a separate field.
Where information is available, review why people declined or withdrew. One person may have had a scheduling conflict; another may have found the task difficult. Do not assume a common cause or remove incomplete attempts because they reduce the reported result. State what is known and what could not be verified.
Separate observations from interpretation
Record what participants actually did, including unsuccessful attempts, withdrawals and requests for help. A positive interview comment is evidence of an opinion, not proof of future purchasing. Distinguish a demonstration from normal use and an invitation from completed participation.
In a hypothetical example for illustration, 8 of 12 participants complete an application without a correction. Check what happened to the other 4; do not assume they all encountered the same problem. That result describes those 12 cases under those conditions. It does not prove that all customers will achieve the same outcome. Review who was excluded, whether staff provided extra help and whether the task matched routine work. A baseline is a recorded result from the previous method used for comparison. Compare against it when available, explaining differences in participants, tasks and assistance. Without a suitable comparison, 8 of 12 does not establish that guided review reduced corrections. Nor does it alone justify continuing without checking the agreed acceptance criteria.
Check delivery and economics before scaling
Estimate the actual staff time, support effort, technology cost and capacity needed for routine delivery. A manually assisted pilot can hide work that will become expensive at scale. Identify which benefits are observed and which remain forecasts. Ask what must change before the service can operate reliably without the original project team.
Consider how a larger population may differ from early volunteers. Test interfaces with existing systems, exception handling and responsibility for complaints. A working prototype is one part of the decision, alongside operational readiness and the ability to maintain the service.
Choose among three defensible outcomes
Combine usage evidence with the review of delivery and cost. Choose the decision these findings support and specify the commitment it creates. Continuing means a bounded next step, not approval for full rollout.
| Decision | Reason | Next commitment |
|---|---|---|
| Continue | Evidence supports the hypothesis within the agreed limits | A bounded next stage with remaining uncertainties |
| Adapt | The need remains plausible but the approach has a specific weakness | A revised hypothesis and test |
| Stop | Evidence contradicts the need or risk or cost exceeds acceptable limits | Close obligations and preserve learning |
Do not change success criteria after seeing the result merely to make the experiment appear successful. If new information requires a different criterion, record the change and treat the new test separately. An inconclusive result may require better evidence rather than immediate expansion.
Turn the result into a specific next test
In a hypothetical extension of the earlier example, suppose participants understood the service but needed extra guidance when uploading documents. The team could test simpler upload instructions while keeping the other steps as consistent as possible. The next question becomes clear: does the need for help decrease? Changing price, design and audience together makes a difference harder to explain.
Select participants to suit the question instead of relying only on colleagues who already know the idea. Testing a service for new users with its designers does not represent those users’ experience. Explain the sample limitations in the review decision. Results from a small selected group are not a reliable promise about the whole market.
Measure the hidden work supporting the trial. Record preparation, questions and error correction as well as time spent using the prototype. The delivery process may need improvement before more users join. Identify who will own that work after the experiment, the resources needed to sustain it and the evidence required for the next stage.
If the evidence does not justify continuing, record what the team learned and what it will not retest without new information. An orderly stop preserves resources and closes commitments. Continuing requires a new question, boundaries and a review date rather than open-ended approval to scale.
Record the learning and next owner
Keep a short decision record containing the original hypothesis, test conditions, evidence, limitations, decision and owner. Preserve useful negative findings so another team does not repeat the same avoidable test. Remove temporary access permissions that are no longer needed and fulfil participant commitments when the experiment ends. Handle data according to approved retention rules.
Review starts with a clear hypothesis and stated acceptance criteria. It ends with a decision supported by evidence and understood limitations. If the result remains inconclusive, identify the missing evidence instead of declaring success or failure without a basis. Give each next step limited resources, an owner and a review date.
The Innovation and Future Foresight service can connect experimentation with strategic priorities and an innovation portfolio: a group of initiatives managed together to allocate resources and balance risks. The purpose of review is a better next decision, not a guaranteed successful innovation.