Data Core · Performance Analysis · Module
Method
How the question will be answered, and whether that method can actually answer it — because most business questions are causal and most available methods are not.
The idea
How it works
A causal question asked of observational data can be answered only under assumptions that are usually false and rarely stated. Naming the assumptions is what separates a careful analysis from a confident one.
Where the question genuinely matters and the data cannot support it, designing a small experiment is frequently cheaper than a large analysis that will not settle anything.
Working with it
In practice
- 01
Match the method to the question type
Descriptive, comparative or causal. Only the third requires an experiment or a careful identification argument.
- 02
Write down the assumptions
What has to be true for the method to answer the question. This is where most analyses quietly fail.
- 03
Consider an experiment
For questions that recur and matter, a small test settles what a large analysis will only illuminate.
- 04
Say what the method cannot do
Before running it, so the limitation is part of the design rather than a caveat at the end.
One level in
The components of method
A component is something that exists afterwards which did not exist before — a deliverable or a mechanism, not an intention.
For questions that recur and matter, a small experiment settles what a large analysis will only illuminate.
The other modules in performance analysis
The question
What decision the analysis is meant to inform, and what answer would change it.
LearnReading the result
What the finding does and does not say, including the alternative explanations — the step where confidence is either calibrated or inflated.
LearnActing on it
What changed as a result, and whether the change worked — without which the analysis was a document rather than an input.
Learn