Our learnings from analyzing 100,000 emails with Outreach Amplify Intent Classification
Reply rates are important, but they don’t tell the whole story.
What? I know - this is sales sacrilege. I’m not saying the way you’ve been measuring success and tracking KPIs is wrong, but there is a better way that you need to know about ASAP.
Case in point — Any sales rep who’s ever gotten an email that looks like this will tell you that not all replies are created equally:
Yes, the reply rate you get is a huge indicator of engagement with your emails. But that’s Sales and Marketing 101. If you want to truly drive next-level results, the most important thing for you to measure is the quality of your replies, with a clear focus on your positive reply rate.
Intent Classification: The Basics
In sales, every reply to an outbound email will essentially fall into one of three categories:
- Unsubscribes - You’re completely out of the running here. You legally cannot continue to cold email this prospect or you will be out of compliance with CAN-SPAM laws. This is the one reply you don’t want in your inbox.
- Objections - Your prospect replied to your email only to let you know that they don’t have budget. Ideal? No. With the right strategy, can you turn this into a deeper conversation? You bet. While not the ideal scenario, there’s still opportunity in an objection.
- Positive replies - This is the magical reply that goes something like, “Funny you should email me because I was just thinking there has to be a better solution than what I’m doing today. When can we meet?” BOOM. Everything you do should be optimizing for this reaction.
Ultimately, you want to maximize the number of emails with positive intent you receive. That’s why simply measuring reply rate just won’t cut it. What you need your sales email metrics to measure is what’s driving the replies you want.
The solution: Measure reply intent in conjunction with reply rate.
Through our machine learning program, Amplify, we were able to gain a new level of insight into our top performing emails, seeing not only what drove replies, but what drove positive replies.
Lessons from Amplify: When Reply Rate doesn’t tell the Whole Story
A common problem sales reps are always trying to solve for is how to tactfully nudge a prospect who has gone dark without being annoying. We tested four emails to try and figure out the best way to do this.
Pop quiz time! Which email do you think is the most effective?
I’ll be completely honest, I initially guessed template 2, which was the worst performing template by just about every metric, so no shame if you’re with me on this one.
But the real lesson here is that if you were only optimizing for reply rate, you would have chosen to use Template 1, which provided a reply rate of 6.1%. However, 20% of those replies were unsubscribes, which means you just lost that prospect forever.
When you dig a little deeper and look at the intent of the replies, you’ll see that in fact Template 3 was the most successful in generating positive replies (which, depending on your team, can convert tens of times better than objections) and very effective at minimizing unsubscribes. In fact Template 3’s positive reply rate was almost twice as high (70% higher to be exact) than the next best performer, Template 1.
Multiply that out over 1,000 emails, and optimizing for positive replies in this case would leave you with 7 fewer unsubscribes and 5 more positive interactions with a higher likelihood of conversion. And that’s just one step in one sequence!
Think of all the qualified meetings and future happy customers you could be leaving on the table if you’re not optimizing for conversations your prospects actually want to have.
Pro tip: Want to know why Template 3 outperformed Template 1? Our experts think the closing line that focused on driving value for the end user (“explore some potential use cases”) sealed the deal. The vagueness of finding “time next week to discuss” in Template 1 may have left prospects wondering what they were discussing and why, causing them to unsubscribe.
How we do it at Outreach
It sounds simple enough, but it’s hard to manually measure and classify intent. Most email systems (ours included!) have a manual sentiment tracking tool, but on average, your reps are classifying only 1% of their inbound emails because they forget or don’t have any more time to spend on non-selling activities.
That’s where machine learning comes in to help you scale up your efforts.
Using Amplify, we trained a data model to identify positive, objection, and unsubscribe intent in email replies. Our machine learning engine used Natural Language Processing (NLP) to automatically classify over a hundred thousand emails by intent, so we could aggregate data to determine what emails were driving the results we wanted at a greater scale, rather than using anecdotal evidence to deduce what works and what doesn’t.
Amplify helped us add the all-important "reply quality" layer to the age-old reply quantity metric to drive for more positive results.
