by Andrew Maliwauki February 17, 2016

5 Ways to Create a Winning A/B Testing Strategy

When it comes to split testing your landing pages, finding a good place to start can often be a challenge.

Whether it’s a brand new page with seemingly endless possibilities or maybe an older one that’s gotten you a little burned out, every test needs to count, so it’s important to have a great testing strategy in place.

Now, most people don’t want to throw money at guesswork, they want results. Fortunately, setting up an effective testing strategy is relatively easy.

Here are 5 simple, tried and true ways to get and keep your A/B testing on track and full of ideas for months and even years!

1. Create a Solid A/B Testing Plan

When building out the first page for a client, it’s easy to get caught up in the initial design, layout and focus of the page and put variants on the back burner.


To make sure that I keep an eye on the big picture, I always develop a plan for my first few months of A/B testing before I jump into the actual design process.

Setting up a detailed roadmap of what you will be testing next changes how you think about your initial page. For example, instead of debating what piece of information to cram on a page, you specifically choose what you want to try on your first iteration and save the rest for a later test.

As you put together your A/B testing strategy, consider what you will want to test—social proof placement, headline ideas, hero shot variations, etc. Once you know what you want to test and in what order, designing pages to test your hypotheses should be easy.

2. Resist the Urge to Test Too Often

When you’re A/B testing, there’s a strange urge to keep creating new variants, especially when your current test isn’t producing the sort of results you’re hoping for.

However, in the world of A/B testing, patience is the key to success. Not many pages (especially those with low traffic to begin with) will start to see results anytime soon after beginning a new test.

When split testing your pages, keep an eye on that confidence rate and don’t fool yourself or your client that a page is performing 200% higher than the champion when it’s only been two days. Give it time to even out.

If you hit 90-95% confidence a couple of weeks out and your new page still has a 200% better conversion rate, then you have permission to throw a party of epic proportions.

3. Test New Changes Incrementally

As I mentioned earlier, it’s important to keep plan what you want to change on your pages for future variants. However, simultaneously testing all your hypotheses makes it almost impossible to make any real progress.


To avoid this scenario, only test a few changes at a time.

How many variants you can test at a time will depend on your traffic. If you’re sending thousands of visitors per month to a page, you can probably test several things simultaneously. If your page only gets ten views per month, you might only be able to pull off one test a year.

4. Never Stop Testing

It probably goes without saying, but testing is a never ending process.

Having a new champion variant simply increases your ability to further increase your conversion rate, leads and sales. Once you’ve discovered a winner, immediately begin thinking about what the next test will entail.

Similarly, there are no failed tests in A/B testing. If your new variant doesn’t outperform your original page, you’ve simply learned that your new idea isn’t better than your old idea.

In either case, if you can figure out why your winner…well…won, you can use that information to improve the performance of your current page and many other pages.


This is often easier said then done, but there are some good practices to keep in mind when this situation arises.

For one, don’t repeat the same thing you just did. If your new headline was reason for the increase in conversions, leave that area alone for now and see what else can benefit your page.

Constantly put yourself in your audiences shoes and ask the questions that they would when browsing your page. These “pain points” will help you to think subjectively about your design and identify elements that might not be working quite right.

5. Keep an Eye On the Competition

This point is important to think about regardless of how your split-testing is going. It’s good practice to keep up-to-date on what other competitors in the industry are doing, and you may even be surprised to discover that they’ve taken some of your ideas and implemented them on their pages!

Competitors are a great way to find new ideas to test. As we just pointed out, they are probably doing the same thing to you. If you see something that makes a lot of sense to you, test it and see if it works for your particular market!

Note, I’d avoid directly copying without testing, though, since you never know how good your competition’s page is performing.


When it comes to conversion rate optimization, there are no set rules for what works and what doesn’t.

However, having a solid A/B testing strategy in place from day one will save you time and stress when coming up with new variant ideas later on down the road. Stick to your strategy and you’ll see remarkable results!

By the way, if you’d like some ideas on how to put together a killer A/B testing strategy, let me know here or in the comments. I’d love to help.

Have any good testing strategy ideas you’d add to this list? Let me know in the comments!

  • Site Optimization

Andrew Maliwauki

Andrew Maliwauki

Andrew is a graphic designer and CRO specialist who adamantly proclaims "There Will Be Blood" as the greatest movie ever made. He also loves pugs. A lot.

Leave a comment

Featured Posts

Background Parallax
Background Parallax

Let's Do This!

Let's get in touch to see if we're a good fit to help you reach your business goals.
Contact the Disruptive Team
  • Google Premier Partner Logo White
  • Hubspot Certified Partner Logo
  • Bing Ads Accredited Professional Logo
  • Facebook Marketing Partner Logo
  • Klaviyo Master Elite Marketing Partner Logo
  • 5-Star Reviews on Clutch Logo
  • Best PPC Rankings Award
    Best PPC Rankings Award
  • Utah Venture Entrepreneur Forum Top 25 Under 5 Company
    Utah Venture Entrepreneur Forum Top 25 Under 5 Company
  • UB Best Companies to Work For Award
    UB Best Companies to Work For Award
  • 2020 National Excellence Award Winner
    2020 National Excellence Award Winner
  • Inc. 5000 Fastest-Growing Companies Award
    Inc. 5000 Fastest-Growing Companies Award
  • Inc. 500 Fastest-Growing Companies #145 Icon
    Inc. 500 Fastest-Growing Companies #145 Icon
  • Top PPC Company 2022 Design Rush Award
    Top PPC Company 2022 Design Rush Award
  • Top 1000 Companies Global 2021 Clutch Award
    Top 1000 Companies Global 2021 Clutch Award
  • Top Digital Marketing Agencies 2022 Clutch Award
    Top Digital Marketing Agencies 2022 Clutch Award

©2024 Disruptive Advertising