Skip to content

Optimización de conversión

What Is an A/B Test, and What Is Worth Testing on Your Website?

Equipo Artisma

Agencia de Marketing B2B

8 min read
¿Qué es una prueba A/B y qué vale la pena probar dentro de tu sitio?

The marketing team wanted to know whether a new headline on the services page would bring in more quote requests. It set up an A/B test: half of the visitors saw the usual headline and the other half saw the new one. By the fourth day, the new version had twice as many requests, and the director asked for it to be published for everyone. The test was stopped that same afternoon.

The requests from those four days were six versus three. Three of the six came from the same company, which requested quotes for three plants separately. Over the next two months, requests from that page dropped, and nobody knew whether the cause was the new headline or a cooler market. The original version no longer existed for comparison.

The company paid for the tool and took up its team's time. In the end it made a decision by intuition, now with a chart to back it up.

An A/B test compares two versions of a page with real visitors at the same time. What is worth testing is whatever drives the buyer's decision: the headline, the contact offer and the form.

Comparing against the previous month mixes your change with everything else

The usual way to evaluate a change on the website of a company that sells to other companies (B2B) is to publish it and compare requests with those of the previous month. Many things change from one month to the next. An industry trade show comes around, a campaign launches or a large customer finalizes its annual budget. Any of those events moves requests more than a headline does, and the comparison credits them to the change.

An A/B test, also called a split test, solves that problem. During the same period, one share of visitors sees the current version of the page, which is version A. Another share sees the version with the change, which is version B. Everything that happens in the market affects both groups equally. The difference left at the end is explained by the change.

Random assignment makes the comparison fair

The testing tool randomly assigns each visitor a version and shows them the same one on later visits. Randomness prevents one group from receiving more mobile visitors, more longtime customers or more traffic from a campaign. If the split is made by day, by region or by channel, the test mixes the change with other causes again.

A hypothesis written down before you start

Every test starts with a concrete, written hypothesis. For example: “If the services page headline names the buyer's problem instead of our product line, more visitors will click request a quote.” The hypothesis sets three things from the start: what changes, which metric decides and why you expect an improvement. Without it, any result can be interpreted to suit whoever reads it at the end.

Four mistakes that turn an A/B test into an opinion with charts

A poorly run test is worse than not testing, because it gives a wrong decision the appearance of data. These are the four mistakes we find most often.

Stopping the test as soon as one version pulls ahead

At the start, the differences between versions swing widely because there are few conversions. A version can be ahead on Tuesday and behind on Friday without anything changing. The duration and the number of conversions are set before starting, and the test is evaluated only once both are met. Stopping it at the first favorable result produces winners that do not hold up later.

Measuring clicks when the business depends on opportunities

A more eye-catching button can get more clicks and bring in the same number of requests. A shorter form can double submissions and bring in contacts that sales discards. The test's primary metric is the one closest to the business that your volume lets you measure. It is reviewed together with the quality of the requests in your customer relationship management system (CRM).

Ending the test before completing full weeks

The B2B buyer behaves differently on Monday than on Saturday, and differently at the start of the month than at the end. A test that runs from Wednesday to Sunday compares days that are not alike. The minimum duration is two full weeks, and most B2B websites need more time to gather enough conversions.

Testing changes so small that your traffic never detects them

The smaller the difference you expect, the more conversions you need to tell it apart from chance. Changing a shade of blue on the button produces, at best, a minimal difference. With the traffic of a B2B website, that test runs for months and ends without a clear winner. The changes worth a test are the ones that change what the buyer understands or what they are asked for.

What is worth testing on your website

To choose what to test, start with the pages that get the most visits and sit closest to the request: the home page, the service or product pages with the most traffic and the contact page. Within them, these are the five elements that weigh most in the decision of an industrial or services buyer.

The headline and the promise on the first screen

The headline decides in seconds whether the visitor keeps reading. A useful test pits two different approaches against each other. One describes what you do and the other names the result your customer gets. Another option is to pit a general headline against one aimed at the industry that buys from you the most. If one approach wins, the lesson also applies to your ads and your emails.

The contact offer you make to the visitor

Requesting a quote requires the buyer to have a clear sense of their need. A 15-minute call, a technical review or a downloadable data sheet asks for a smaller commitment. Testing the main offer of your call to action changes how many visitors take the first step and at what stage they reach sales. On a B2B website, this test has more impact than any design adjustment.

The form and the number of steps

The form is the last obstacle before the request. You can test a four-field form against an eight-field one, or a single screen against two steps with a progress bar. This test is measured all the way to the CRM, because a shorter form attracts more requests and also changes their quality.

Trust signals next to the button

The B2B buyer hesitates right before submitting their details. One version can show, next to the button, the sectors you serve, your certifications or a guarantee with clear terms. If that version wins, you know how much trust weighs in your sale, and the rest of the service pages get the same treatment.

The order of the service page

A service page can open with the buyer's problem or with the technical description of what you offer. The order changes who keeps reading. Testing both structures with the same information shows how your market prefers to get informed, and that lesson guides the rest of the website.

When your traffic falls short, change the design of the test

Before setting up a test, count how many conversions the page you want to test receives each month. A sample size calculator estimates how many you need based on your conversion rate and the improvement you expect to detect. If the result demands more months than you can wait, adjust the test with these four decisions:

  • Test on the page with the most traffic on the website and leave lower-volume pages for later.

  • Use an intermediate step with more volume as the primary metric, such as the click on the quote button, and review requests as a control metric.

  • Test big changes, such as a reorganized page or a different offer, so the difference can be detected with fewer conversions.

  • Test the message first in your ads or emails, which get more volume, and bring the winning approach to the website.

If the volume still falls short, conversion rate optimization offers other ways to measure rigorously, such as comparing equivalent periods before and after the change. The discipline of writing the hypothesis and setting the metric stays the same.

What to do with the result, whichever version wins

When the test ends, there are three possible outcomes. If version B wins clearly, it is published for everyone and the lesson is applied to similar pages. If version A wins, the change is discarded and the reason the hypothesis failed is documented. If the two end up even, you keep the version that is simpler to maintain and test a bigger change.

Every result goes into a test log with the hypothesis, the dates, the volume and the decision. That log prevents repeating tests that have already been run. It also gives your team a knowledge of the buyer that no opinion in the boardroom can replace. A result holds for your market at that moment, so key tests are repeated after a redesign or a major change in your offer.

At Artisma, we leave analytics active from the launch of every website. In the Contenido and Crecimiento plans, we apply one improvement per month based on the metrics report. When the page's volume allows it, we validate that improvement with an A/B test. When it does not, we validate it with a comparison of equivalent periods that we document with the same rigor.

Frequently Asked Questions

Does an A/B test affect my website's search engine rankings?

A properly configured test preserves your rankings. Both versions are shown for a limited period, and search engines see the same content as people do. If version B lives at a different address, that page tells search engines which one is the original. When the test ends, the winning version is published and the other is removed.

How long should an A/B test last?

As long as it takes to gather the number of conversions you calculated before starting, always in full weeks. The practical minimum is two weeks, to cover every day of the week. If the calculation points to more than two or three months, adjust the test with another page, another metric or a bigger change.

What is a multivariate test, and does my website need one?

A multivariate test combines several changes at once, for example two headlines and two images, and measures each combination. It splits traffic among many versions, so it needs much more volume than an A/B test. On most B2B websites, a sequence of well-chosen A/B tests teaches more in less time.

If your website changes based on the opinion from the last meeting, every change is a bet nobody measures. Schedule a free 15-minute call and we will review with you which page has the volume for a test, which change to test first and how to measure it all the way to the quote request.

BOOK YOUR CALL

Pick a time