UI and UX Optimization · Testing changes · Component
The measure
What the change is meant to improve, agreed in advance — because deciding afterwards means the winning metric is chosen from the results.
The deliverable
What it is
A test with several measures and no primary one always produces a winner, because something moved. That is not evidence.
The measure and the effect size that would matter should both be written before the change is built.
One level in
What it is made of
Each element is a constituent part of the component. Follow one to see the attributes it carries.
The effect size
How much improvement would matter.
3 attributes: Minimum effect · Reason · Value if achieved
Learn
A test with several measures and no primary one always produces a winner. That is not evidence.
The other components in testing changes
The power
Whether the available volume can detect the effect that matters — because an underpowered test produces noise presented as evidence.
LearnThe result
What happened, including the nulls — because the nulls are the majority and are what keeps expectations calibrated.
Learn