UI and UX Optimization · Testing changes · Component
The result
What happened, including the nulls — because the nulls are the majority and are what keeps expectations calibrated.
The deliverable
What it is
The honest finding of most testing programmes is that the majority of changes produce no measurable difference. Recording the nulls is what keeps that visible.
A programme reporting only wins is a programme with a publication bias, and its win rate is meaningless.
One level in
What it is made of
Each element is a constituent part of the component. Follow one to see the attributes it carries.
The guardrail check
Whether anything else got worse.
3 attributes: Guardrail · Movement · Acceptable
Learn
A win rate around a quarter is healthy. A reported rate near one means only obvious changes are being tested.
The other components in testing changes
The measure
What the change is meant to improve, agreed in advance — because deciding afterwards means the winning metric is chosen from the results.
LearnThe power
Whether the available volume can detect the effect that matters — because an underpowered test produces noise presented as evidence.
Learn