Skip to content

NSIInsights

How to measure whether a business decision worked

Define success before acting and compare the same measure across comparable conditions. Record the baseline, outcome window, eligible population and other changes. Improvement after an action is an observation, not automatically evidence that the action caused it.

Published by North Star Intel

Define the baseline and outcome window

The baseline is the starting measurement, not the target. Record its source, dates and inclusion rule before examining the outcome. The target says what you hope to achieve. Separating them prevents a favorable result from changing the question after the fact.

Choose a window long enough for the intended effect to appear, and define its end in advance. Scheduling may affect this week. Purchasing may need a replenishment cycle. If the window changes, preserve the original plan and record why.

Compare the same denominator

A count needs context. Twelve late jobs out of sixty eligible jobs differs from twelve out of two hundred. Define eligibility consistently, including cancellations, duplicates and missing timestamps. Keep the raw count beside the rate so a small population remains visible.

Fictional calculation: a route trial

Fictional example, not a client result: the baseline has 12 late jobs out of 60 eligible jobs, or 20%. The outcome window has 8 out of 80, or 10%. The observed rate falls by 10 percentage points. That is not a 10% relative reduction and does not prove causation.

The second period also used an extra vehicle and had fewer long-distance visits. Those differences could explain part of the change. The defensible conclusion is an observed improvement with unresolved alternative explanations, not a claim that the route trial alone produced it.

Keep a confounder log

Record staffing, weather, demand mix, price changes, outages and changes in measurement that could affect the result. A comparable location can help test an explanation, but is not automatically a valid comparison group. Different customers or capacity can still distort the comparison.

When inconclusive is the right answer

What if the baseline is missing? Do not reconstruct certainty from memory. Name the gap and collect a prospective baseline. What if the sample is small? Keep uncertainty visible and avoid broad generalization. What if the result worsens? Use the same measurement rules as for improvements.

A review sheet needs the decision, expectation, baseline source and window, outcome source and window, eligible denominator, alternative explanations and next check. Inconclusive means the available evidence cannot support the desired conclusion. A checked signature does not remove that limit.

Sections

Insights

Reading on operational decisions, accountability and evidence.

Back to Insights

Find a page

Get started

Make your next operating decision more accountable.

Tell us the decision you need to prioritize. We will confirm the scope, available evidence and next step before proceeding.