A/B Testing - Send to Winner

Being able to easily conduct A/B tests is a key activity for the emarketing professional. With the Send to Winner feature, Informz has made this so easy to use you have to give it a go!

When you do an A/B test, you usually split your target group to see which one responds better to whatever you are testing. This could be different subject lines, friendly froms, stories, graphic density, font size, etc. This feature let's you automatically send the better performing version of the mailing, to the remainder of the target group outside of your test segment. The only thing you have to do is decide how long you want to wait after doing the test and what you want to base determination of "the winner" on.

Steps

After specifying the type of A/B test you want to do on the Set-Up tab of the mailing, go to the Send tab.

Click Schedule Mailing.

Select the schedule option of Once and choose an initial send time.

Select Send to Winner in the delivery method selections.

Percentage

Each A/B version of the mailing can be set to a percentage of subscribers, anywhere from 5% to 40% in increments of 5%. After the winner is determined, the rest of the targeted subscribers will receive winning version.

For example, if the user selects 20%, then 20% of the target group receive Version A of the mailing, and a different 20% receive Version B. The winning version is calculated, and then the remaining 60% of the target group receives the winning version.

Timing

A date and time must be included in the Send to Winner delivery setup. A Date/Time widget appears when the field is clicked, allowing the user to set a date and time. On that time and date, the final tallies are made for the two A/B versions, then winner is selected, and the mailing is sent to the rest of the target group.

The time and date selected must be at least 4 hours after the initial publication of the A/B test. This ensures that enough time has passed to get feedback for a winner.

Determining the Winning Version

Organizations may have different views as to what constitutes the best metric for indicating success. Informz gives you 4 categories to pick from:

  • Highest Delivery Rate
  • Highest Open Rate
  • Highest Click Rate
  • Lowest Unsubscribe Rate

Other Considerations

  • A mailing has to have at least 20 subscribers in order to use A/B Send to Winner. If it doesn’t, the mailing just uses A/B Testing as normal.
  • If there is a tie between the A version and B version, the A version is sent to the remaining subscribers.

Final Thoughts

A mailing that uses A/B Test is still a regular mailing. It is not a triggered mailing.