How to Create Sustainable A/B Testing Process

Ask anyone in online marketing about A/B testing and you will find out that everyone has their own theories on what does and doesn’t work. These theories are usually like snowflakes in that no two are the same.

In and of itself, that’s fine. But the problem is, not many designers or PPC account managers have a set testing process that they follow to brainstorm, create and launch successful A/B tests.

Now, you might think, is that really a big deal? After all, they’re testing, right? Isn’t that what matters?

Unfortunately, no. Truly great testing is more than just trying things at random because you have some great new idea or you’re hoping to “get lucky.” The goal of great testing is to learn about your audience and use what you learn to improve the conversion rate of your page or site.

With that in mind, let me outline the process I use for putting together an A/B test. Hopefully, if you’re like most online marketers and don’t have a defined process for your testing, this will give you a good place to start.

Essentially, there are 4 steps to my testing process:

Step 1: Establish a Great Baseline

In my opinion, this is the most important part of any successful A/B test. Before you start testing, your page already needs to be functional and converting at an acceptable rate.

Sometimes clients tell me that they want to run an A/B test because their page isn’t converting. To me, that seems less like conversion rate optimization and more like a repair job.

Think about it this way, if your car is broken, will tuning the engine do you any good?

You have to get the car running and then you can optimize its performance.

The same idea applies to A/B testing. The first part of my testing process is to take a look at the page, review its current results and see if the baseline design is adequate. If the page needs a massive overhaul just to get it running properly, I focus on getting the page in working order before I start testing.

Step 2: Review Data and Make Minor Change

Once your “A” page has had plenty of time to gather visitors and provide some valuable data I like to analyze the findings and plan a minor change that I feel will make major results. Some of the things I like to try first are moving a form up on a page, changing a CTA or changing a headline.

Now, you’re probably wondering, how do I know what to change just by looking at numbers?

It’s simple. For example, if phone calls are your best type of leads and you have a page that produces calls at a decent rate, you might consider adding more call CTAs to the page.

Headline changes are another minor change that can result in major conversion results. One easy way to optimize your headlines is to talk to your sales team and find out what kinds of questions they regularly get from leads.

If they get a lot of questions like “How much is it?”, “Do you have any testimonials?” or “Can you tell me more about…”, you’ve got some really good potential headline material to test.

Once you have sifted through your data and made your major minor change, you are ready to launch your first A/B test. Set both variants to 50% and drive some traffic.

Step 3: Review Data and Create Version 2.0

After running your test for 7-15+ days (depending on traffic, of course), there’s a good chance that you’ll have an obvious winner.

Now, based on what you learned from your first test, it’s time to try making a bigger change to the page design, imaging and/or messaging.

For example, if you talked to your sales team, found out that most of your leads are asking questions about pricing and created a winning landing page with a headline that addresses pricing, you may want to try redesigning your whole page to be more pricing focused.

You may need to try several different tests here to really get things dialed in. Remember, with each test, you want to learn something about your audience, so use what you learn in each test to direct your next test.

The goal here is to change up the experience based on data gathered from the client’s account but this is not a redesign. That comes next in our testing process.

Step 4: Review Data and Change Up the Experience

At this point, I’m usually killing it with my tests. I have great data and a landing page that is gathering leads like a kid at Halloween.

At this point, it’s time to change up the experience completely.

Why? Well, remember, we started this whole testing process with the assumption that your baseline page was well-designed for your target audience. So far, we’ve done a lot to improve the performance of that page, but we’re still dealing with that same fundamental assumption.

Now it’s time to challenge that assumption.

Take the data you’ve acquired throughout your tests, start from scratch and create a very, very different variant. For example, you might try going from a dark page to a white page, completely changing your messaging, adding or removing functionality…challenge everything!

This can be a big risk to take, but if you use all that awesome data to guide your decisions, a complete overhaul like this can improve your page performance in a big way!

Conclusion

Now, you may or may not choose to follow my testing process, but if you want to be successful with A/B testing, there needs to be a method to your madness. The key to great A/B testing is patience and consistency.

And the best way to do that is to create (and write out) a well-defined testing process.

By the way, if you’d like me to help you come up with a testing process for your own site and pages, let me know here or in the comments! I’d love to help.

What does your testing process look like? How do you improve the performance of your landing pages and site?