Creating a Successful A/B Split Test

By Jenn Alessi

02/09/15

Do you ever find yourself debating between two different subject lines, images, or calls to action on your email campaign? Instead of blindly choosing the winner, why not create an A/B split test? You can use the information you learn from the test to fine-tune your email campaign and assist with future campaign decisions. Follow these steps to get started and produce a successful A/B split test:

1. Determine what you want to test. Although the change might be small, it can make all the difference in your results. You might want to test:

  • Subject lines
  • From names
  • Week, day, or time of send
  • Calls to action (button size, color, copy, location)
  • Headlines
  • Images
  • Design

2. Decide which metric will determine the winner. Depending on what you are testing (decided in step 1), use one of these metrics to measure your success.

  • Open rate
  • Click-through rate
  • Conversion rate

3. Determine the specific question you’re looking to answer. This will help you focus on the end goal of your campaign. Some examples might be:

  • Which subject line produces a higher open rate?
  • Which call to action produces the highest conversion rate?
  • Which headline creates a higher click-through rate?

JennGraphic4. Only test one variable at a time. If you test multiple variables, it will be extremely difficult to determine which variable had the highest impact on your results.

5. Randomly divide your list into two equal segments. Most email service providers have built-in features that will do this for you. If not, make sure your groups are random, including a mix of engaged/unengaged subscribers. Assign one variation to one list segment and another variation to the other list segment.

6. Determine the winner and send your campaign. Again, most email service providers have built-in A/B split test functions, so this part could be automated. Based on your winning criteria that you determined in Step 2, send the winning campaign to the remainder of your list.

7. Compare similar A/B split tests to each other. If you are constantly testing subject lines, compare your results. You might determine that subject lines with questions in them perform better than statements. You can use this information to perfect future campaigns.