Swing for the Fence with these A/B Testing tips

Guest post by Patricia McLaren, Co-founder and Head of Customer Operations at CopyShoppe

I recently read Scott Leese’s book on successful sales mentality, “Addicted to the Process,” and the opening analogy struck my attention: A baseball player in a slump.

When a player’s in a slump, they place extreme pressure on themselves… sounds like some sales reps out there. They need to hit a home run. They become obsessed with the struggle instead of focusing on success itself. To hit that home run, they tweak things - but too many things at once. They change up their batting stance, chase every breaking ball, and swing at the first pitch every time.

These frantic changes don’t work, whether we’re talking about home runs or email campaigns.

If the baseball player controlled those tweaks, however, he could achieve success a bit faster. We have to adapt and change our messaging approach if we want to see different results, and that’s where A/B testing comes in. It’s a constant art, a yin and yang of email copy, to figure out what’s working and how to scale it. But there’s a fine line between changing too much too soon, and running controlled A/B tests that drive more results.

Here’s how you can hit a home run with your A/B tests:

Start at Home Plate

In other words, establish your baseline. Typically, your baseline is your current result (your control) and you can test different variables to improve on that result. If you’re looking to improve your email open rates, and it’s currently at 30%, decide on a modest improvement goal, such as a 3% increase, to work toward (and partner with CopyShoppe to help you get there). Then decide on the most important variables that will help you get there - in this case, subject line variations would most impact your open rate.

Tips on subject lines to improve email open rate:

• Use customization. You’re catching their attention for the first time, so use personalization beyond their name, if possible. What other data points can you leverage? Ex: technographic data, competitive information, partner network, or internal referrals.
• Keep them short and sweet. We often see that one- or two-word subject lines perform better than a long sentence. Ex: “Message from {{competitor}}” or “Advice on this”
• Avoid clickbait. People can smell it from a mile away, so be sincere and relevant to the actual message in your email.
• Test often!

Concentrate on One Pitch at a Time

In email copy terms, focus and control one variable at a time. It is possible to test more than one thing at a time, but it’s best to run a tight experiment in order to fully attribute results to a specific variable. If you’re testing subject lines for better open rates, the email copy should remain constant.

Variables to test:

• Subject lines
• Calls to action - Change up the content and use action verbs beyond “connect” and “sync”
• Send schedules - Do your prospects engage with you on the weekends?
• Tonality - Play around with witty, conversational, educational, or formal approaches
• Link sharing - Provide documents or case studies and track their click rates

Strength in Numbers

The more times you practice your swing, the better you’ll do during the actual game. When you’re running an A/B test, the more data you have the stronger (and faster) your results will be.

Let’s say you put 30 people randomly into subject lines A and B. You’re basing the results from a selection of 15 people, and there are too many external factors that could have influenced 15 people to click your email. Instead, continue the experiment and keep activity high until you have a substantial sample size of at least 100 contacts. The more activity, the faster the performance data can prove the test winner. Then you’ll be able to hit those “out of the parks” more often and more confidently.

Tips on activity:

• Dedicate specific team members to run the test and give them incentives to follow through.
• Commit to adding a specific number of prospects into the test daily to keep activity high.
• Consider bounce backs or deliverability rates - You’ll need actual sends to generate reliable data.
• Plan realistic timelines to reach your ideal sample size.

Don’t Switch Your Stance Mid-Swing

Trust your basics! Follow through fully if you’re hitting a home-run, or conducting an A/B test. If you switch things up mid-test, your results will be skewed. How will you rely on the test results if you’ve made any changes to the initial parameters? It can be difficult for us to wait out an entire A/B test in an age run on instant gratification, but be patient and controlled until the performance data is strong enough for you to properly scale the winning result.

Email campaigns take time to perfect. Just like any skill that needs improvement, your A/B tests take patience, control, and follow-through. Plan your goals and testing details ahead of time, control each aspect during the test, and be patient for the performance data to speak for itself. Sales communication is evolving quickly, and our messaging has to change with it. Always test your email copy to see consistent improvement, and soon enough you, or your team, will be hitting your messaging out of the park every time! Test away A-team!