Strategy-Led Innovation Playbook · Evidence

Evidence and Metrics

A number only helps if you know how much it proves. This page shows how to judge evidence, which metrics matter in each phase, and how to use rules of thumb without fooling yourself.

01 / Judge

Evidence has two parts. The weaker one decides.

Commitment: how much the customer actually gave up. Words cost nothing. Time costs something. Money costs more. Coming back and paying again costs the most.

Reliability: how well the test could have told you that you were wrong. Who you tested, how many, whether you watched behavior or collected opinions, and whether the pass/fail line was written before the results came in.

Strong evidence needs both. Two hundred pre-orders from friends and family show high commitment but low reliability, so the evidence is weak. One enthusiastic interview with the right customer is reliable but shows little commitment, so it is weak too. Judge each result by its weaker part.

02 / Commit

The commitment ladder

Six steps, from nothing to proof that people keep paying. Each step costs the customer more than the one before, which is why it tells you more.

Step	What it looks like	What it can show
1 Nothing yet	No contact with customers. Only your beliefs.	Nothing. This is where every idea starts.
2 Said so	Interviews, surveys, "I would buy that", "I would pay".	Interest, and early signs the problem exists. Never willingness to pay.
3 Gave time or effort	Showed you their workaround, shared real data, sat through a long session, tried a rough prototype.	The problem is real enough to spend effort on.
4 Signed up or signed a letter	Email signup, waitlist, letter of intent.	Intent with a small cost. A letter of intent from a named payer is a strong Solution signal. It is not a payment.
5 Paid	Money changed hands: a purchase, a pre-order, a deposit.	Willingness to pay.
6 Kept paying or using	Renewals, repeat purchases, use that holds over time.	Product-market fit, when it holds across cohorts.

Willingness to pay is shown only by payment. A pre-order or a deposit is a payment. A survey answer is not.

03 / Test

How reliable was the test?

Who: people in the segment you named, or friends, colleagues and fans?
How many: enough that one enthusiastic person can't swing the result?
What you observed: behavior you watched, or opinions you were told?
The line: was the pass/fail line written down before the results came in?
The other way: could the test have come out against you? If not, it wasn't a test.

04 / Track

What to measure in each phase

Phase	What you are deciding	Metrics that matter
Discovery Problem, then Solution	Is the problem real, frequent and costly? Does the segment prefer your solution, and does a named payer have a reason to pay?	Risky assumptions named and ranked. Hypotheses with pass/fail lines. Experiment results. The commitment step reached: Gave time or effort for the problem, Signed up or signed a letter for the solution. Cost and time per learning. Learning velocity.
Incubation Paying, then Keeping	Do people pay, and do they keep paying or using it?	Activation rate and time to activation. Share of active users who pay. Retention and churn by cohort. Repeat purchase.
Commercialization/Scale Go-to-market, then Scale	Can we sell repeatedly at acceptable cost, then deliver at volume while it still serves the strategy?	Acquisition cost by channel and its payback period. Revenue growth, revenue per user, lifetime value against acquisition cost. Referral rate. Operating data at volume.

05 / Define

Definitions

Acquisition

Cost per acquisition = acquisition spend ÷ new customers. Conversion rate = acquired ÷ visitors × 100. Track it by channel and by traffic type.

Activation

Activation rate = users who complete the key action ÷ new users × 100. Also track time to activation and drop-off at each step.

Retention

Retention = users at period end ÷ users at start × 100, by cohort. Churn = users lost ÷ users at start × 100.

Revenue

Revenue per user = revenue ÷ users. Lifetime value ≈ revenue per user ÷ churn rate. Share paying = paying users ÷ active users × 100.

Referral

Referral rate = referrals ÷ users × 100. Viral coefficient = referrals per user × conversion of referrals. NPS = % promoters minus % detractors; supporting evidence only.

Learning

Cost per learning, time per learning, learning velocity (assumptions tested per month), and checkpoint velocity: the time and cost to move from one checkpoint to the next.

06 / Calibrate

Rules of thumb

These are Exponential Scout rules of thumb, not published benchmarks. Use them as a starting point, state the traffic type or context, and adjust for your market.

What	Rule of thumb	Context
Landing page signup	2 to 5% of visitors	Cold, targeted traffic. Varies by industry.
Landing page signup	Around 15% of visitors	Warm traffic: people who already know you.
Email signup to trial	15 to 25%	People who signed up on the landing page.
Trial to paid	15 to 20%	People who started a trial.
Early adopters	100 engaged early users	Before you claim traction.
Problem interviews	80% confirm the problem	Said so. Look for workarounds to move up the ladder.
Solution interest	60% say they would pay	An interest signal only. Not willingness to pay.
Sean Ellis survey	40% "very disappointed"	Supporting evidence of fit only. Retention and payment lead.
NPS	50 or more	Supporting evidence only.
Organic growth	20% or more month over month	Without paid marketing. Supporting evidence only.
Lifetime value to acquisition cost	Above 3 to 1	Commercialization.
Monthly churn	Investigate above 5 to 10%	Subscription businesses.
Onboarding drop-off	Simplify above 30%	Any step of the activation funnel.
Share of active users who pay	Review pricing below 10%	Freemium and usage-based models.
Learning pace	At least 3 tests a month; one learning cycle a week or faster	Discovery and Incubation.
Checkpoint pace	Review if a checkpoint takes more than 3 months	Any phase.

Set your own pass/fail line before the test.

07 / Commit to the line

Write the pass/fail line before the test runs

Name the assumption you are testing and why it is risky.
Write down what you will measure, the line that counts as a pass, who you will test, how many, and by when.
Share the line with your team or backer before any results come in.
Run the test. Compare the result to the line, not to what you hoped.
Decide: persevere, pivot, retest, or stop. If you move the line after seeing the results, the test does not count.
If you stop, write a stop record: what you tested, what you learned, and what you will do with it.

A line written after the results is a story, not a test.

Next: start with a Foundation Sprint, then test in Discovery.