Definition: A/B Testing
Running an online store is associated with constant changes. Sometimes it is enough to add or remove products from the catalog, and sometimes you need to optimize the page layout. When it comes to optimizing your website, the decision is not always obvious. This is where the A/B testing comes in handy.
The A/B testing involves examining different versions of a company’s website (e.g. its layout, marketing emails) on two groups of customers and checking which of these groups generates better results. The winning variant is then implemented on the site. In today’s post, we’ll tell you how to plan an A/B test, what elements are worth testing, and what to keep in mind.
Planning A/B Testing
We distinguish two basic categories of changes that can be introduced in the company on the basis of tests. We can test elements on-site (i.e. the company’s website) or off-site (e.g. advertising, marketing e-mails). The purpose of the tests is the same in both cases: to improve the company’s effectiveness in terms of increasing sales.
Planning ahead is important. It is worth deciding in advance which elements will be tested. It’s rather obvious, but it’s really not worth testing every little bit of a page. It’s good to focus on those that generate clicks and – more importantly – conversions.
It is also worth considering what effects we expect. You need to know the current results to be able to compare them with the results of the changed items. In some tests, revision A is the current process and revision B is a new feature. You can also assign new elements to both versions A and B to compare both with the current solution. If you are testing several new items, make sure at least one of them performs better than the current solution before replacing the original. Otherwise, it will of course be wiser to keep the current elements.
Basics of the A/B test.
Two versions of the same page are displayed to the same number of people. The one with the better result “wins”.
What to test?
Optimizing your website is a great idea. Some changes can really significantly increase your sales – and that’s what it’s all about. Before you make any changes (or test those changes), let’s take a look at a few things you might want to consider testing.
Headlines: If your business employs a content marketing strategy, headlines attract visitors to your page and make them click through. From an SEO perspective, this is also very important. Find out what helps improve your store’s position in search results. This will help you find out if more people are clicking your ads, links, and posts.
Nouveau call-to-action
CTA buttons: CTA (Call-to-Action) buttons can do wonders. Usually, they are placed under the relevant content, but they can also be used to add a product to the basket. They also allow you to attract potential customers. It all depends on what you’re using them for, but overall they’re of huge importance for conversion. Their appearance (an image or a simple button?) And their location (top, middle, or bottom of the page?) Can also make a huge difference.
Content: Like headlines, the content of your blogs, pages, and emails can help you improve your store’s ranking on Google. Relevant content can also induce a visitor to make a purchase. A/B test of two versions of the content can pay off.
E-mails: Speaking of e-mails, it is also worth doing an A/B test on them. Whether or not the recipient opens the message is largely determined by the subject. If your mailing list is quite long, consider sending the same email with different topics to see which is the most effective (same for newsletters ). Likewise, you can test the content of your emails to see if one style appeals to your customers better than another. Oftentimes, there are also CTA buttons in emails.
Website: This is obviously a serious matter. Testing different items may produce different results. On the home page, it’s a good idea to change your header image to see if your bounce rate improves. It’s also a good idea to rearrange the menu buttons. If you’re opening a website in a new country, test different variants to see how the new market responds. The field to show off here is huge, so choose the elements that you think are the most important.
Ads: As with most of the items mentioned above, small changes can make a huge difference to your results. You may think that a certain image works great for your Facebook ad and the results show something different. We used to test two photos with each other – mine and a man, shirtless and sprawled on the couch, playing with a tablet. Needless to say, it was surprising (and sad) to me that the shirtless guy did (much) better than me. In addition to the images, you can also change the text and duration of ad serving.
Target audience: If you run an online store, you likely use advertising on Google or Facebook. Both services offer great targeting options for your target audience. If you run a relatively new company, you may not know your target group yet. Not only that, you may think you know her and be wrong (entrepreneurs really often have a wrong opinion about it). Testing different target groups can produce surprising results.
A/B testing showed that a guy who looks almost like me (left) received far more clicks than the real me (right)!
It’s been a hard week for me …remember, In addition to knowing what to test, there are a few other things to keep in mind.
Do not rush
How long should the A/B test take? This is perhaps the most common question we ask ourselves when determining their effectiveness. If your site has a lot of traffic, you may be able to get some pretty accurate results pretty quickly (for example after a week or so), but there is a good chance you will have to wait a few weeks (two to six) for this to happen. There are some tools (more on this later in this article) that can help you determine how many people are needed to run a test in order for the results to be reliable.
Likewise, it works the other way around – the tests cannot take too long. Why? Sometimes different variables can influence the results. For example, I do not recommend A/B testing during the Christmas period, as shopping habits are slightly different around Christmas (this does not apply, of course, to a situation where you want to test “Christmas” page elements). Other variables that can affect outcomes over time include:
- Season
- Holidays
- Ads displayed during the test
- Mentions about the company in the press (positive or negative)
Be careful with assumptions
As you conduct your experiments, you will surely make some assumptions about their results. It’s understandable that you want to be right about your predictions. However, some sellers unknowingly end testing too early once they have achieved the results they expected. So I repeat: it is very important to run the tests long enough for the results to be reliable.