Here are some quick tips on how to make better decisions on your AB tests with statistics
CONTEXT:
When you run an experiment or analyze data, you want to know if your findings are "statistically significant". When making decisions on effectiveness and engagement of marketing activities, you want to make sure your results aren't due to chance instead of the variable you're testing on (example: message copy). Running a chi square test is recommended and can be done without pulling out the good old pencil, paper or dusty TI-84 from high school. Here's a reliable handy chi-square calculator.
HERE ARE THE STEPS:
Determine the metric. In this example I'll pick CTR
Calculate your CTR (clicks / sends) for the two test variants you're looking to compare. (In this screenshot below I did this based on 20% vs 25% CTR)
Plug those numbers into the two top boxes
Click "Result"
In this example I need 1504 sends in each group of my test. Because I picked a power of 0.8, I can be 80% sure that group 2 will result in higher CTR than group 1