Operating Cadence
Part of Startup metrics and management information
Keeping metric definitions stable between reviews
Document metric counting rules, check comparability and show definition changes without hiding breaks in the series.
Keep a metric comparable by recording its counting rule before the first review and reusing that rule. If a better definition is needed, date the change, explain its effect and identify which earlier figures remain comparable. A familiar metric name does not guarantee that two figures mean the same thing.
Key Principles for Stable Metrics
- Record counting rules before first review
- Re-use for consistency across reviews
- Date and explain definition changes
- Identify which prior figures remain comparable
- A name doesn’t guarantee same meaning
- Two figures with same name may differ by definition
Write a definition someone else can reproduce
For each regular measure, record the event being measured, eligible population, exclusions, reporting period, date assigned to each event, calculation, source and owner. Specify how cancelled work, reopened cases and late entries are treated when they could materially affect the result.
Take a hypothetical measure called “on-time handoffs”. It is incomplete until the team agrees what a handoff is, whether the original or revised due date applies, what counts as received and which cases are eligible. Two colleagues could otherwise produce different figures from the same work records without making an arithmetic error.
A compact definition record can contain:
- Name and purpose:
- the decision the measure informs.
- Calculation:
- numerator, denominator where relevant, and units.
- Coverage:
- included and excluded cases.
- Timing:
- reporting period, event date and data cut-off.
- Source and owner:
- where the inputs come from and who resolves questions.
- History:
- effective date and reason for a rule change.
Keep it short enough to maintain so another person can reproduce the figure.
Check before comparing periods
Ask the owner whether the offer, workflow, source system or data-entry practice changed. Check that the same population is included. A rise in completed jobs could reflect a new class of small jobs.
A fall in recorded late cases could reflect entries that had not arrived by the cut-off. Put a material coverage gap beside the figure rather than treating it as a business improvement.
Show the break when a rule changes
If the old rule no longer answers the decision, write the proposed replacement and apply both rules to a suitable set of records. Identify the cases that switch classification. Choose an effective date and mark the break in the series.
If earlier data can be recalculated reliably, label the recalculated figures and retain the original reported view. Otherwise, start a new series instead of drawing an uninterrupted trend.
Suppose the team previously counted a handoff as complete when sales sent a form. It now wants completion to mean delivery received all required inputs. Both measures may be useful, but they answer different questions.
Placing last month's figure under the first rule beside this month's under the second without a note invites a false comparison.
Correct a data error separately from changing a definition. Record what was wrong, which periods were affected and whether prior reports were amended. At the next review, the reader should be able to tell whether performance changed, the method changed or the evidence is still incomplete.
