Ashley Zeckman

Convert More Customers: Tips & Tricks For Running A/B Tests On Your Email Campaigns

Ashley Zeckman     Email Marketing, Online Marketing

Email continues to be one of the most effective online marketing tactics but many companies struggle to establish benchmarks or successfully improve performance. How many people are receiving, reading and acting on email messages? Those are all key questions to improve email marketing performance.

Email open rates can vary depending on the target industry and even the job function of the prospects.  Testing to improve email marketing performance is essential, but determining which variables have the largest impact on the overall effectiveness of an email campaign can seem overwhelming.

However, a process driven approach to identifying success factors that lead to increased conversions will set your company up for success both short term, and long term.

One way of improving your email marketing process involves testing: A/B or multivariate testing, to assess which tactics lead to the desired outcomes such as click throughs and conversions.

What is A/B & Multivariate Testing?

Put simply, A/B testing involves sending out different versions of any email campaign to segments of your existing customer or subscriber list.  Typically A/B testing includes changing only one element at a time, whereas multivariate testing includes multiple elements.  There are many variables to consider when running an email test so it is best to pick a maximum 3-4 items to test for each campaign.  If you are unsure about which variables to test please reference the list below for guidance.

What Can Be Tested?

As mentioned you don’t want to get carried away with testing too many variables at once.  Below I have included some of the tactics that your team may want to consider when running a A/B or multivarite email test.

  • Time of Day
  • Day of Week
  • Subject Lines
  • Body Copy
  • Layout
  • Calls to Action
  • Design (including images)
  • Personalization
  • Offer

Sample Email Test

Who Should A/B Email Test Be Sent To?

If you search online or talk to marketing experts you may receive a variety of answers.  Depending on your list size you will want to send out A/B tests to a large enough segment to collect enough data to make an informed decision.  I would recommend splitting your list into three parts:

  • Recipients of test A – ¼ of your list
  • Recipients of test B- ¼ of your list
  • Recipients of the “winning” email – ½ of your list

What Should Be Tracked?

  • Open Rate
  • Click Through Rate
  • UnSubscribe Rate

A Best Practice Approach For Running A/B & Multivariate Tests for Email Marketing

Data Doesn’t Lie: Base  your course of action on proven numbers, not which email you BELIEVE will perform better.

Use Tools: Services such as MailChimp provide a tool for running A/B tests.

Test Often: The way that your audience reacts to information will evolve, make sure that your process evolves as well.

While we talk a lot about search optimization, social media and content marketing here at Online Marketing Blog, make no mistake: Email Marketing is a powerful communication and conversion tool. It integrates well with other online marketing efforts to both attract and retain customers. Remember ABO: Always Be Optimizing. It applies to email marketing performance just as much as pay per click or SEO.

I’m curious to know what tests your company may have run in the past.  What were your results?  How did these results shape your process going forward?

PoorSo SoOKGoodAwesome (1 votes, average: 4.00 out of 5)

Ashley Zeckman About Ashley Zeckman

Ashley Zeckman is the Senior Director of Digital Strategy for TopRank Marketing. In addition to finding innovative ways to showcase the exceptional work of her team, she is also responsible for creating digital marketing programs that drive customer acquisition and growth for the agency. Her background includes expertise in everything from content marketing strategy to branding, account management and social media.

Outside of her role at TopRank Online Marketing, Ashley prides herself on being a great home chef, painter, prankster and dog mom.


  1. Avatar Sharynsheldon says

    Hi Ashley,

    I’ve only been split testing optin forms right now and know that I’ll have to do the types of tests you’re talking about soon. So, I’m curious to know your opinion of what size a list needs to be to be “big enough” for this type of A/B testing.

    • azeckman azeckman says

      There’s no “magic number” that we can share – it is really dictated on the size of your distribution/audience and how your communications have performed historically that drive how large your sample would need to be to clearly identify your winner.

  2. I have finished testing on the days of the week and there seems to be more clickthroughs over the weekends especially in the afternoon. I will run another A/B for the weekdays to using the body content, subject lines and call to action on different times and monitor clickthroughs and conversions. We shall continue with the debate thereafter. 

    • azeckman azeckman says

      Great to hear you’ve already been running some tests! Please keep us updated on the results of the other tests you plan on running in the future.

  3. Avatar Sourabh Sharma says

     YOU have done a great work its really amazing …….. Thnaks for Sharing this ..
    Advertising Agency

  4. Avatar DaveMarinaccio says

    This is horrible advice. You don’t test multiple variables simultaneously, ever. You’ll never know what part is working or not that way. One thing at a time, always.

    • azeckman azeckman says


      While I appreciate your opinion I must disagree. Standard A/B testing which involves varying only one element at a time can be a very time intensive and exhausting process, endless even. What testing multiple variables will allow your team to do is:

      1) Get through the testing phases much faster
      2) Determine which combinations of email elements produce the best results

      Thanks for sharing!


  5. Is MailChimp free to use?

  6. Avatar NewClientsEachMonth says

    As you know, split test everything!  We tend to change one variable at a time, I know it’s slower but it’s certainly working.  Thanks for the great article!

  7. Time of Day and Day of Week are important factors for having a successful email marketing and i liked that you explain more about them because these are really important topics in addition to running  tests on Email campaigns.

  8. This post is well-timed for me.  I’m going to split test my email marketing shortly.  I really appreciate the points you made.  Thanks.

  9.  Email
    is one of the most effective means of communication to customers, but
    it can be hard to tell what exactly is prompting your customers and open
    and read your emails. I used OfficeAutopilot software to automate my
    emails, and the program calculated for me the open rates and the
    following actions of my customers. Utilizing the techniques from this
    article to test my customers’ responsiveness helped my business
    enormously. I created various versions of my emails and sent them out to
    different customers. Then, I was able to use OfficeAutopilot’s
    “if-then” conditional setting to automate the proper follow-up action
    depending on how my customer responded to my email (ie. if my customer
    opened up my email, then the customer would be marked as an interested
    customer). All these techniques helped me convert my leads into actual