Fluxx Tech
Every website argument ends the same way: someone’s opinion wins. A/B testing replaces the opinion with a number. You show version A to half your visitors and version B to the other half, and see which produces more enquiries.
It is powerful, and it is misused more often than not. The main misuse: running tests on sites that do not have enough traffic to produce a trustworthy answer.
How it works
1. Pick one page and one thing to change
2. Create version B
3. Traffic is split randomly between A and B
4. Both run at the same time, to the same kind of audience
5. Measure the conversion rate for each
6. Keep the winner, if the result is reliable
Running A for a month and B the next month is not a test. Seasonality, campaigns and weekdays all change the result.
What to test first
Biggest impact, in rough order:
Element | Example |
Headline | Benefit-led versus descriptive |
Call to action | “Get a quote” versus “WhatsApp us” |
CTA placement | Sticky mobile button versus one at the bottom |
Form length | Five fields versus two |
Price display | Showing prices versus “request a quote” |
Proof | Adding reviews or client logos |
Hero image | Product versus people versus work in progress |
Page structure | Order of sections |
Headlines, CTAs and form length produce the largest effects. Button colours produce blog posts.
Sample size: the part that decides everything
A test needs enough conversions, not just enough visitors, before the result means anything. A page with 300 visits and 6 enquiries a month cannot produce a reliable answer in a week, or in a month.
Rough guidance: you want a few hundred conversions in total across both versions for a modest difference, and far more to detect a small one. Use an online sample size calculator with your current conversion rate before you start. If the calculator says you need six months, do not run the test.
Also: run for at least one full week, ideally two, so you cover every weekday. Weekend and weekday visitors behave differently.
Reading the result honestly
• Do not stop early because B is ahead on day three. Early leads reverse constantly.
• Do not peek and stop at the moment the result looks good; that is how you get false winners.
• Decide the test duration and sample size before starting, and stick to it.
• Accept “no difference” as a valid result. Most tests produce it, and that is useful information.
• A 2% difference on 80 visitors is noise, not a finding.
Common mistakes
• Testing with too little traffic, which produces confident nonsense
• Testing several changes at once, so you do not know what worked
• Running A and B in different periods
• Stopping as soon as the result looks favourable
• Testing trivia instead of the headline, offer or form
• Not checking the mobile experience of version B
• Forgetting that the test script can slow the page or cause a visible flicker, which itself affects results
• Not recording what you tested and what happened, so the same debate restarts in six months
When your traffic is too low
Most small business sites are in this position, and that is fine. Alternatives that work at low volume:
Approach | What it gives |
Heatmaps and session recordings | Observed problems, no statistics needed. See our guide |
Exit or on-page surveys | Why people did not enquire, in their words |
User testing with five people | Watch five real users attempt a task; most usability problems surface |
Before-and-after with a long measurement period | Weaker evidence, but better than nothing |
Copy fundamentals | Clear headline, visible price guidance, obvious CTA, short form; these usually beat whatever is there now |
Ad-level testing | Ad platforms have far more volume than your site, so test messaging there first |
That last one is underused. Test two headlines as ad copy, where you get thousands of impressions quickly, then put the winner on the page.
Keep a test log
For every test: hypothesis, what changed, dates, sample size, result, decision. Within a year this becomes the most useful marketing document you own, and it prevents re-running tests you already answered.
Frequently asked questions
How long should a test run? Until you reach your pre-calculated sample size, and at least one to two full weeks.
Can I test on a page with 500 visits a month? Usually not meaningfully. Use qualitative methods instead.
Does A/B testing hurt SEO? Not when implemented properly with standard testing tools and the same content substance on both versions. Do not show different content to search engines than to users.
