13 September 2025 · 4 min read
Calculate the detectable effect first
With your traffic and baseline conversion, work out the smallest effect you could detect in a reasonable time. If that is a twenty percent lift, only test changes plausibly that big.
Running an underpowered test produces a coin flip with a confidence interval.
Test upstream where volume is
Test on the highest-traffic step of the funnel. Testing a page that four hundred people see per month will never conclude.
Move the test, not the timeline.
Use qualitative methods for small changes
Session recordings, usability tests, and support ticket analysis find problems that low-traffic A/B tests cannot measure.
Five moderated sessions often beat an inconclusive month-long test.
Where to go from here
If you are weighing experimentation for an upcoming project, the useful next step is usually a short scoping conversation rather than a longer article. Bring your constraints — budget, timeline, existing systems, and the team you already have — and the decision usually resolves itself in an hour.
We run these as a fixed-scope discovery session with a written recommendation at the end, including the cases where our advice is that you do not need us.
Working on something like this?
Send us the brief. We will tell you what we would build, what we would cut, and roughly what it costs.
Talk to us