A/A Testing

What is A/A testing?

A/A testing is a way of double-checking the results of any A/B testing platform by split testing between two identical versions of a webpage. Essentially, this means you’re showing the exact same webpage to two groups of people.

In theory, the A/A test should result in no difference between the control and the variant versions.

A/A testing vs. A/B testing

Most people working in ecommerce are familiar with A/B testing. In A/B testing, you show two groups different versions of a page, while with A/A testing you show both groups an identical version.

Difference between A/A and A/B testing

Why is A/A testing important?

If you’re feeling unsure about whether the results of your A/B testing are statistically significant or you have doubts about whether your A/B testing tool is working properly, A/A testing can answer your questions.

If you see a significant difference in results between the two identical versions of your webpage, it means you have one of two problems. Either your testing tool isn’t functioning properly, or you don’t have enough visitors going through your testing platform to reach statistical significance.

How to do A/A testing?

Although the exact process depends on the tool you use, it’s generally simple to run A/A testing.

1. Create two identical versions of the same page/content

First, you need to choose which webpage you’re going to test. This can be your homepage or a landing page. It should be a page that gets a lot of traffic so that you can get a good sample size.

2. Identify your Key Performance Indicator (KPI)

This can be responding to a call-to-action, entering an email address, or actually making a purchase. 

3. Divide the group

Now you divide your incoming traffic in half, sending one half to the control and the other half to the identical variant.

4. Track the KPIs for both groups

As you’re literally testing the same page, the results you get shouldn’t differ (at least not significantly). If they do, you should reevaluate your A/B testing strategy.