Market Core · Marketing Channels · Module
Testing and incrementality
Whether a channel contributes demand that would not otherwise have arrived, established by withholding it rather than by observing it.
The idea
How it works
Observed performance and incremental contribution are different quantities. A channel reaching people who were already going to buy reports excellent numbers and adds nothing, and no amount of tracking distinguishes the two.
A holdout answers it. Withhold the channel in one region or one period and compare. It costs some demand and it is the only method that produces an answer rather than a model.
Working with it
In practice
- 01
Test the largest lines first
The channels with the most spend have the most to gain or lose from the answer.
- 02
Withhold long enough to see the effect
Short holdouts measure noise. Most channels need several weeks before the difference is readable.
- 03
Decide in advance what result changes what
Agreeing the response beforehand prevents an inconvenient result being explained away.
- 04
Accept that some channels add nothing
It is a common finding and an expensive one to ignore.
One level in
The components of testing and incrementality
A component is something that exists afterwards which did not exist before — a deliverable or a mechanism, not an intention.
Agree what each possible result would change before running the test. Afterwards, inconvenient results get explained away.
The other modules in marketing channels
Channel selection
Which channels are used and why — matched to where the audience actually is and to what the message requires, rather than to what is available.
LearnChannel economics
What each channel truly costs and returns, including production, management time and the effect on other channels.
LearnThe mix
How channels combine and in what sequence — because channels interact, and a mix managed as a set of independent lines will misread all of them.
Learn