Data Core · Data Collection · Module
Data quality
Whether the data is complete, accurate, timely and consistent — measured rather than assumed, since the assumption is always that it is fine.
The idea
How it works
Data quality is discovered rather than measured in most organisations: someone acts on a number, the result is wrong, and an investigation finds the cause. Measuring it in advance is considerably cheaper.
Completeness is the dimension that most often fails and the easiest to check. A field populated in sixty per cent of records is producing analysis about the sixty per cent.
Working with it
In practice
- 01
Measure completeness per field
What proportion of records have it. The result is routinely worse than assumed and takes minutes to obtain.
- 02
Check accuracy against reality
Sample records and verify them against something external. Internal consistency is not accuracy.
- 03
Fix at source, not in the report
A cleaning step in a report fixes one output. Fixing collection fixes all of them.
- 04
Publish the quality figures
A number reported alongside its completeness is used more carefully than one reported alone.
One level in
The components of data quality
A component is something that exists afterwards which did not exist before — a deliverable or a mechanism, not an intention.
Report a figure alongside its completeness. A number known to rest on sixty per cent of records is used more carefully.
The other modules in data collection
The sources
Where data actually comes from, and which source is authoritative where several disagree.
LearnInstrumentation
What is deliberately captured and how — the design decision that determines which questions the organisation will ever be able to answer.
LearnConsent and basis
What the organisation is entitled to collect and hold, on what basis, and what it has told people.
Learn