A/B testing is a powerful tool that allows you to evaluate the most effective mix of factors for engaging with your audience. By alternating copy, images and more, you can zero in on the best way to optimize your SMS performance.

Voyage’s A/B testing feature enables you to easily create multiple variants of a Campaign and experiment with different variables to optimize your SMS performance.

Here are some variables you can consider testing:

Note: While there are multiple variables to experiment with, it is best to test one variable at a time. Otherwise, it becomes difficult to determine what specifically drove better performance.

When creating a Campaign, you will see the option “Add A/B Testing” above the message composer.

When you click this button, a new variant will be created and you can edit each variant separately. You can create a maximum of 4 variants per Campaign.
​

Once you’ve successfully created your variants, you will then be able to review and adjust the following settings for your A/B test.

Scheduled Start Time: this determines the date and time of when the variants will be sent (i.e. when the A/B test begins).

Test Audience: this determines the amount of subscribers that will be exposed to the A/B test, which works as a function of the number of subscribers in your Audience.

For example, if your Audience contains 5000 subscribers and you select a Test Audience of 40%, then the variants will be sent to only 2000 subscribers.

The winning variant will then be sent to the remaining subscribers in your Audience i.e. 3000 subscribers.

Test Duration: this determines how long the A/B test will run for before determining the winning variant.

For example, if the Test Duration is marked as 2 hours and the Scheduled Start Time is set to 10:00 AM, then the winning variant will be determined at 12:00 PM and it will automatically be sent to the rest of your Audience.

Once you’ve configured your A/B test settings, you will see a summary recapping the most important information before launching your A/B test.

The winning variant is determined by the variant with the highest click-through-rate (CTR).

For example, if you create an A/B test with two variants and Variant 1 has a higher CTR than Variant 2, then Variant 1 will be selected as the winner and automatically sent to the remaining portion of your Audience.

The winning variant will be determined at the end of the Test Duration and will display a green trophy in the stats table.

It is important to ensure that you run your A/B tests on large enough audiences to guarantee statistically valid results. A common mistake is to run A/B tests on very small sample sizes and draw conclusions that aren't statistically valid.

In short, determine your necessary sample size in advance using tools such as this chi-square calculator.