A/B Testing: All You Need to Know

Marketing can be hard – you need every edge you can get to stay ahead of the competition. That’s why you need to do A/B testing – it allows you to try out different marketing approaches and see if the alternative yields a better outcome. In this article, you’ll learn all you need to know about A/B testing.

What Is A/B Testing

A/B testing, also known as split or bucket testing, is to test two different strategies to target a group.

For illustration, let’s say you have a newsletter subscription button – it’s called control (the existing element).

You want to know if it’s something can be improved by changing your newsletters in order to gain more subscribers.

So, you make another newsletter with a different subscription button – the placement, the color, the call-to-action copy, the shape, etc. This new element is called a variant.

Now, you show them both to a 50/50 split of your whole audience.

This way you will gather the analytical data from both control and variant which you use as the deciding factor. Which one can reel in the more new subscribers?

The newsletter with more success should be your choice.

It’s just the tip of the iceberg! You can use A/B testing for various marketing platforms, such as social media, visual marketing and much more.

On top of that, it’s still possible to compare more than two elements. This may even give you the much-needed data for better decision-making – the more data to compare, the better predictions you’ll make.

For example, you can add the third newsletter subscription button as the additional variant. You will need more traffic because you will split them into three instead of two.

But, if you have more than three elements, you need to familiarize yourself with multivariate tests.

Why Is A/B Testing Important?

If you want to fix a problem or optimize your existing well-functioning elements (ads, call-to-action, button placement, etc), you need to explore other methods to see if it can yield better results.

In short, A/B testing will serve as a data-driven problem-solving method based on statistical measurement. This will allow companies to be better-informed in making decisions regarding their marketing strategies, websites, apps and so on.

There are many areas which companies and marketers alike can solve and improve by conducting A/B testing. Keep in mind that you need to match the right element that has the most impact on each stat. Those areas are as follows.

  • Conversion Rate – A/B testing will allow you to see if the alternative converts more visitors into buyers than the original method
  • Bounce Rate – you can test out modifications such as page navigation to see if it makes your visitors stay longer or not
  • Click Rate – after making certain changes in your site, you can measure if your visitors have more tendency to click on certain links

What’s more, there is no end to the variety of industries that will benefit from conducting A/B testing. From ecommerce to travel, big companies or small, A/B testing is always a good approach to fixing problems and improving features. Take a look at how each industry benefits from A/B testing.

1. Ecommerce

The homepage of Amazon.comEvery online store owner aims to get as many sales as they can. When faced with low income, you can conduct A/B testing to see what can be improved.

  • Low conversion – it’s common that not all visitors will buy your products. So, how to improve the conversion rate? First, identify the current strategy you have. If you charge them for shipping, you can battle it with marking up the price and providing free shipping.
  • Unattended cart – your customers abandon their carts a lot. What you can do is try to find the cause – does your cart recovery email have with engaging copywriting and clear call-to-action? Is the tone of your customer support lined up with your audience? Test out different methods for each problem and see if it’s better or worse.
  • Product page navigation – it may not look important, but how visitors navigate from one product to another can go a long way in racking up much more sales. For example, if your current method is showing alternative products (e.g. different brand of toothpaste), try showing other items that people usually buy together (e.g. toothbrush and towels).

Ultimately, online store owners will know which strategy works better in increasing sales thanks to A/B testing.

2. Media & Publishing

The homepage of The Huffington PostThere are also many aspects of media and publishing industries that A/B testing can make better. The goal is to capture potential readers’ attention and maximize the chance of them buying your printed goods – and even registering for a paid membership on your site.

So, what are those aspects? Take a look below.

  • Covers – an excellent cover should be able to captivate your readers in one glance. For example, you can tinker with the headlines, the font (type, color, and size), and the background – different models may yield different results.
  • Layouts – bad placement of paragraphs and elements can hurt your readers’ eyes. There are so many you can experiment on – alignment, size, illustration, color, etc.
  • Columns – chances are, your site or printed goods don’t only talk about one subject. So, you should test it out whether your readers like fewer and broad columns or many categories with more specific topics. Also, see if adding a new column – or even erasing one – can boost your sales and increase readers’ satisfaction.
  • Marketing Network – either traditional or digital, some of the methods to promote your magazine or newspaper can be improved – explore different approach such as new areas of stalls, set an alternative tone and style for your social media campaign, etc.
  • Prize Quiz – it’s in everyone’s nature to get their hands on free stuff. That’s why conducting a good prize quiz is one aspect any media company should never overlook. Tweak its difficulty level, the number of questions or actions needed to take, the incentive, etc.

After getting the much-needed data to analyze, you can take the steps that can increase the overall experience of your readers.

3. B2B (Business-to-Business) Tech

The homepage of Markable SolutionsEither you offer digital marketing, security, or management system for other companies, you need to take care of every aspect that leads to more sales and bigger conversion rates.

Make sure you take the necessary steps to build trust with your audience. So, let’s see how A/B testing can achieve that for you.

  • Landing page structure – it can take many forms and there’s no surefire way to pinpoint which aspect in your landing pages may perform better – unless, you run A/B testing on it. Test out different colors, information placement, buttons, and everything else. Throw away the variables with worse results and stick with the good ones.
  • Registration form – it’s one of the most important aspects of your page. If the forms are too cluttered, it may cause potential customers to flee. Factor in the design – make it neat, eye-pleasing, and concise. Try different buttons and call-to-action copies and see which one results in a better conversion rate.
  • Mascots – it’s closely related to the tone of your message. Different audience behaves differently, so you need to test out whether the specific mascot can perform better.

The examples above are just the beginning – there are many other aspects you can test out such as whether to emphasize the word “FREE!” if you offer a free consultation, different PPC placement, navigational layout, and many more.

4. Traveling

The homepage of TravelocityWhen your travel agent services don’t gain enough profit, you can rely on A/B testing to point out the problems for you.

It can be many things – inconvenient purchase experience, unattractive pricing display, and many more. Here are some ways A/B testing can improve aspects in travel sites.

  • Incorrect information warning – your customers may mistakenly skip a column or put the wrong country code for phone numbers. It may frustrate them if they’re shown an error right after they finish filling out the form and click submit. Try different ways – make it appear right away, or spread the columns into different sections.
  • Decoy effect – if you have various travel packages for your customers, you can display them side-by-side to instill a certain perception in customer’s minds. For example, showing three packages with different pricing may lead them to choose the middle package – especially if you add the “best deal” statement on it.
  • Loss Aversion – applying additional motivation can be a good idea. Pressure them into racing to stay at your hotel by showing “1 room(s) left!”. See if it can bump your sales counter. Also, don’t forget to consider the placement – either on the product overview or on the product page.

In other words, experiment on elements that can be improved and weight in your customers’ behavior.

How to Conduct A/B Testing?

Every run of A/B testing can be different depending on your current situation and the problems that you’re facing. Still, there are general steps you can take to conduct A/B testing, which are as follows.

Step 1: Gather Initial Data

Before you do anything, you must first know the conditions of your website or mobile app. Google Analytics is one of the most reliable tools you can use to gather all the needed data – keep the information in arm’s length.

After that, think of certain problems you want to solve or conditions you want to improve such as high bounce rate, low sales, etc.

Step 2: State the Goals

Goals are important so that you can match the problems you have with the relevant aspects.

For example, if you want to test out different pricing displays, then the metrics you should track are conversion rates.

Plus, focusing on a certain goal will prevent you from distractions while testing.

Let’s say you need to increase your traffic by testing different headlines. However, it’s possible that one of them also increases the bounce rate – don’t base your decision on this rate.

Step 3: Formulate A Hypothesis

If you have the proper goal and the correct variables to test out, then you probably have an idea on what a certain variable will achieve.

Let’s say you change the color of a button to increase the number of click-throughs. Then, you may predict that a certain color will perform better – because it lines up with your audience better, or it resembles a certain feeling (such as blue for trust and confidence).

Step 4: Create A Variation

For A/B testing to work, you need version A (the existing one) and version B (the alternative you create).

Make sure to make both versions identical in every other aspect – except for the one distinction you want to test out. Base the challenger on the hypothesis you make then pit them against each other.

For instance, compare the effectiveness of two websites: the one that has the blue call-to-action (variant) and the one with the red button (control).

Step 5: Run the Test

When running the A/B test, there are some factors that you should consider to make it as effective as you can.

  • Test both versions at the same time  – remember, it’s the aspects that you want to try out, not the timing. If you test version A in January and version B in February, who’s to say the results are different because of the different elements – not because of the different timing?
  • Split your audience fairly – in order to truly find the champion, you must put up a fair fight. Divide the audience randomly and evenly so that the results are unbiased.
  • Factor in the duration – if you run the test for a short time only, the test results may not be enough to give you a conclusion.
    The size of traffic you have plays a significant role in how long the test should be running.

Now that this step is taken care of, wait until the results are ripe for the taking.

Step 6: Analyze the Results

After getting the results, you may be inclined to start using whichever version is the winner. However, the process isn’t finished just yet – you still need to determine whether it’s not a false positive by making use of A/B testing calculator.

The calculator will decide if the results will yield the desired impact on the real situation or not. If the confidence level is high, you can choose the version with better results or run another test with a different variant.

Things to Avoid

It’s not uncommon for A/B testing to have errors. That’s why there are recurring mistakes that webmasters around the globe must try to avoid such as:

  1. Rushing the initial data collection – either you don’t wait long enough or you skip the calculation part – consider taking the results with a grain of salt
  2. Relying too much on feelings – A/B testing is done for a reason, which is to get the relevant data to be used as decision-making factors. So, make sure to check the statistical significance of your data and track the right metrics
  3. Running multiple tests – it’s not advised to test two or more elements at once. If you want to test out different buttons that lead to higher conversion rate, don’t test another element that leads to the same goal such as cart recovery email. When the conversion rate becomes higher, it’s hard to determine which element causes that.
  4. Wrong timing – make sure to understand the nature of your testing subject. Some aspects may fail if you take too long or too short.
  5. Not doing follow-ups – when your testing fails, don’t give up! Think of it as a journey, not a destination. Make use of the previous test result and try again.
  6. Wrong tools – there are many A/B testing tools to use – but only a few that you can trust. Consider using OptinMonster and MonsterInsight.


We have covered everything you need to know about A/B testing – the definition, the reasons for it, and how to do it.

To sum up, let’s take a quick look on how to conduct the test once more.

  1. Gather initial data – use Google Analytics to collect information about the platform you’re testing on
  2. State the goals – it will keep you from being distracted – don’t suddenly decide to use the variant because if it makes the bounce rate lower if your goal is increasing conversion
  3. Formulate a hypothesis – make a prediction of how a certain element may perform better
  4. Create a variation – following your hypothesis, create a variant to pit against the existing element
  5. Run the test – remember to factor in the duration, the tools, etc. Make sure to test both versions (control and variant) simultaneously to maintain the validity of the results
  6. Analyze the result – put the data in A/B testing calculator to make sure if the results are significant. Then, choose whichever version performs better.

If you’re a marketer that seeks a method to improve your leads and grow your business, take no further look than A/B testing.

The process may take dedication and thoroughness, but the result is worth your time. So, act right now and get a taste of improvement!

The Author


Ananda Ganesha M. / @ganesha

Ananda Ganesha Maaruf is a digital content writer who loves WordPress and hosting. For years he worked on WordPress projects and then combined that experience with his writing talent to educate Hostinger's users. Aside from writing, he often finds himself searching every nook and cranny for the best all-you-can-eat buffet deals.

Related tutorials

Leave a reply




This site uses Akismet to reduce spam. Learn how your comment data is processed.

Become a part of Hostinger now!

More in Marketing, Website
Facebook Marketing – a Beginner’s Guide