Split Testing

Split testing is a method whereby you show two or more variants of a web page, email, advert or other digital element simultaneously to different groups of visitors. You measure which variant performs better against a pre-selected objective, such as conversion, click-through rate or sign-ups. Traffic is randomly distributed across the variants to ensure a fair comparison. In practice, small and medium-sized companies often test headlines, buttons, price tables or form fields. The aim is to use data to substantiate which approach works better, rather than guessing or making decisions based on gut feeling.

How does split testing work in practice?

You start with a baseline version, often the current page or email. In addition, you create one or more variants in which you adjust exactly one element: a different call-to-action, a different colour scheme or a different order of information. A tool such as Google Optimise, VWO or Optimizely automatically splits your visitors between the two versions. One half sees variant A, the other half variant B. All other conditions remain the same: the same traffic source, the same time, the same target audience. After a statistically significant sample size has been reached, you compare the results. The variant with the highest conversion rate or the lowest bounce rate wins. You implement that version permanently and, if necessary, start a new test on a different element.

Why split testing emerged and why it matters now

The method originates from direct marketing, where mail-order companies have been testing different versions of letters for decades. With the rise of web analytics, the same approach became applicable in the digital sphere. Whereas you used to have to wait weeks for results, you can now see within days or even hours which variant performs better. For small and medium-sized companies, this is valuable because it allows you to optimise in a data-driven way even on a limited budget. You don’t need to invest in a complete redesign if a simple tweak to your ‘order’ button already yields a 15 per cent increase in conversions. In a competitive market, that makes the difference between growing and standing still.

What split testing offers small and medium-sized companies

An online shop with 2,000 visitors per month and a conversion rate of 2 per cent generates 40 orders. If you use split testing to increase the conversion rate to 2.5 per cent, you’ll generate 10 extra orders per month without any additional marketing budget. That’s a direct increase in turnover from the same traffic. The method also works outside e-commerce: a B2B service provider tests two versions of a quotation request form and discovers that a shorter version yields 30 per cent more submissions. In search engine optimisation, you often combine split testing with landing pages to measure which message best matches the search intent. In this way, you gradually build a better user experience, based on behaviour rather than assumptions.

You should apply split testing as soon as you have sufficient traffic to draw reliable conclusions. In practice, this means at least several hundred visitors per variant. The method works best for elements that directly influence your conversion goal: buttons, headings, price displays or forms. Below are four specific situations in which small and medium-sized companies use split testing, plus an overview of when the method is or isn’t appropriate.

Landing pages for advertising campaigns

If you’re buying paid traffic via Google Ads or social media, you want every click to deliver a return. A common mistake is for companies to direct all adverts to the same general homepage. By creating a separate landing page that matches the advert copy exactly, you increase its relevance. With split testing, for example, you can compare two versions: one with a video explanation and one with a step-by-step guide in text form. You measure which version generates more enquiries or downloads. It often turns out that a shorter page with a single clear call-to-action works better than a long page full of information. This way, you get more out of the same advertising budget. When designing a website, you can incorporate these insights directly into the structure of new pages.

Email campaigns and newsletters

Email marketing stands or falls on the open rate and click-through rate. Split testing allows you to compare two subject lines, two senders or two call-to-action buttons. For example: an online course provider tests the subject line “Last chance: 20 per cent discount” against “3 places left for the October intake”. The second variant achieves 18 per cent more opens, because scarcity appeals more strongly than a discount. The same applies to the position of buttons: at the top of the email versus at the bottom after the full text. By testing systematically, you build up a playbook of what works for your target audience. This playbook is invaluable for future campaigns and eliminates the need for guesswork. Link your email tool to your CRM to test segments in an even more targeted way.

Checkout flow in online shops

Online shops lose an average of 70 per cent of their visitors during the checkout process. Split testing helps you identify bottlenecks. For example, test a checkout requiring account registration against a guest checkout. Or compare a one-page checkout with a step-by-step flow. A furniture shop discovered that removing the ‘telephone number’ field resulted in 12 per cent more completed orders, as customers found it intrusive. Another commonly used test involves displaying delivery costs: early in the process versus only at the final step. Transparency upfront often reduces bounce rates, even though this may seem counter-intuitive. When developing online shops, we build in testing capabilities as standard, so that you can continue to optimise after launch.

When split testing is the right choice and when it isn’t

Split testing works well if you have sufficient traffic, a clear conversion goal and want to test a single variable. It doesn’t work if you have too few visitors: with 50 visitors a week, a reliable test takes months. Nor does it work if you want to adjust several elements at once, as you won’t know which element made the difference. And it won’t work if you have no idea what you want to improve: in that case, start with heatmaps or user research to identify bottlenecks. Split testing is a refinement tool, not a magic bullet. Use it if you already have a working foundation and want to improve step by step.