Costs
Part of Reading digital customer journeys with a sceptical eye
Reading case studies and digital customer journeys evidence: what to check before you fund a test
Published journey case studies are selected evidence, so extract the mechanism, rebuild that journey locally, and grade the proof before you fund it.
What to take away
- A published journey account is sponsor-selected evidence. It carries a mechanism, not a forecast for your traffic.
- Copy the mechanism, the state model and the test design. Rebuild the denominator and baseline from your own analytics.
- Complete a seven-field record before you release budget. An empty field is the decision.
- Run one mechanism per test. Name the owner, the guardrail metric and the window in advance.
- Set the local test window before launchfour to eight weeks, or 300 to 500 completed outcomes per variant.
Map the states before you copy anything
The GOV.UK Service Manual describes creating an experience map from research with a range of users. It records what people do, think and feel over time, and where the service breaks.
Teams that skip the failure state repeat known traps. Digital customer journeys mistakes lists them, and the list is cheap to check before you accept a case writer's stage names.
That broken state is the part that moves budget. A map listing stages is a poster. A map naming the broken state, its evidence and its owner is a work order.
A named case and what it supplies
Baymard Institute curates published checkout research and reports an average documented cart abandonment rate near 70 percent across the studies it tracks. It also states that better checkout design can lift conversion by about a third on a large commerce site.
Take the first figure as a benchmark of a known friction. Take the second as a sponsor claim. Neither says what your checkout will do.
The mechanism travels: forced account creation, error messages that never resolve the problem, and costs shown late at payment.
Four extraction patterns for published cases
Four extraction patterns
| Pattern | What the case supplies | What your analytics must supply |
|---|---|---|
| Behavioral error repair | Shape of one repeatable failure | Size of the population it hits |
| Mixed-method checkout | Task observation and log evidence | Funnel baseline and segment sizes |
| Technical signup failure | The failing transition and its cause | Error rates and any new failure mode |
| Online-to-offline match | Authority, match rule, loss rate | Matched outcome rate, unmatched records |
[figure 1]
Four extraction patterns
Pattern
- Error-repair
- Shape of failure
- Mixed-method checkout
- Method mix
- Signup-failure
- Expected transition
- Online-to-offline
- Match rule
What the case supplies
- Error-repair
- Population size
- Mixed-method checkout
- Limits per method
- Signup-failure
- Segmented cause
- Online-to-offline
- Loss rate
What your data supplies
- Error-repair
- Mixed-method checkout
- Signup-failure
- Online-to-offline
Reading a case through one pattern keeps the sponsor's claim separate from your own work. Digital marketing strategy case studies applies that reading to published results and names the operating conditions behind them.
Record each method's limitation beside its result, because a small observation sample and a large funnel both mislead when quoted alone. A lead is not the outcome. The completed business result is.
Online-to-offline results turn on the match rule. Keep unmatched records rather than deleting them, and write down the retention period. California's CCPA gives consumers rights over that data, so the record should say why you hold it.
The fixed record to fill first
Seven fields, written before the funding conversation.
The fixed record
- Claim as published
- Exact wording, source, date
- Denominator and baseline
- What was counted, measured against what
- Period and window
- Test start, test end, days after release
- Population and context
- Who, where, under which rules
- Mechanism
- The single thing that changed
- Evidence grade
- Verified, sponsor-reported, unverifiable
- Local decision
- Test, defer or reject, plus an owner
[figure 2]
Evidence record fields
- Actor and scenario
- Starting problem and baseline metric
- Data sources and research sample
- Journey states and the intervention
- Comparison group and metric definition
- Period, population and reported result
- Sponsor, and the evidence that is missing
Keep the source wording separate from your reading of it. Reject any percentage with no denominator, no baseline or no outcome window.
A published number becomes usable only when a comparable local number sits beside it. Journey benchmarks shows which peer figures are matched closely enough to compare.
Sizing the local test
Typical planning range, not a rule: a single-mechanism funnel test needs about four to eight weeks live, or roughly 300 to 500 completed outcomes per variant, whichever comes later. Low-traffic sites stretch the window instead of shrinking the sample.
- Write the baseline and the outcome metric before launch.
- Change one mechanism and hold the rest of the journey still.
- Name the owner and the guardrailrefund rate, support contacts, page errors, consent rate.
- Read the guardrail first, then the outcome.
- Record the result in the same seven fields.
Grade the published evidence
The FTC policy statement on advertising substantiation requires a reasonable basis for objective claims before they are disseminated. A vendor result repeated in your own materials becomes your claim, so grade it as verified, sponsor-reported or unverifiable, then take your facts to counsel.
In Canada, the same test design runs under PIPEDA and CASL consent rules for email and remarketing. The mechanism does not change. The consent record does.
Apply that discipline to every account you cite. Reading published journeys with a sceptical eye walks through the checks that separate a described mechanism from a borrowed number.
