Opening Up About Email Subject Lines Testing

What is my all-time favorite marketing ops question?

“So, what is our open rate for emails?”

What is my all-time least favorite marketing ops question?

“So, what is our open rate for emails?”

On the surface, analyzing email open rates — and the subject line testing that drives them — is a chance to dive into a metrics conversation over hero data that pretty much everyone can understand. But it’s also a descent into a detailed, nuts-and-bolts explanation involving context, context, context, and why there’s no one single answer to the question.

Breath.

As someone who gigs for a company that helps customers send great emails and who spends “happy place” time coming up with killer mails, I thought I’d share a few hard-won lessons I’ve learned from our own recent email campaigns.

Lesson 1: Always Consider the Audience for Email Subject Lines

Googling the latest B2B email open rates gives you broad benchmarks, but only getting intimate with every corner of your actual database will paint you the real story.

Sure, you marketing badasses, shoot for a 24.88 percent industry average open rate, but don’t fret if you fall way short on occasion. Maybe it’s your offer. Maybe it’s your audience weary of email carpet bombing over the past weeks. Maybe it’s your mundane webinar titles. Maybe it’s your (lack of) wit. Or maybe it Just. Doesn’t. Matter.

Now back to the question, “What is our open rate for emails?”

To set the context, I like to default to our best and brightest audiences: customers and engaged prospects. You might opt for newsletter subscribers. Or buyers opted in to daily deals or offers. Leave for another discussion the multitude of other audiences and secondary streams: nurture drips, event follow-ups, blog digests, product announcements, triggered service notifications, et al.

Case in point context-wise, since the beginning of the year, I’ve seen everything from a ginormous 59 percent open rate for an event follow-up (to a small, but highly engaged audience) to anemic 11 percent open rates for a non-rising pizza-related promo (to target prospects).

Mini-lessons:

  1. Start by focusing on the response of your most valuable audience(s), likely the subject of constant subject line testing — measure results campaign to campaign to see where your latest falls.
  2. Don’t be satisfied with a simple aggregate open rate metric from your email provider or marketing automation system — source by audiences, e.g., customers vs. prospects, to put results in context.

Lesson 2: Don’t Worry About Keeping Email Subject Lines Simple

More often than not, I’ll find the most basic approach to announcing an upcoming webinar or event is the simple-and-direct route: who, what, where, and why. Like this for a January 2018 webinar, featuring date and title:

[Webinar, 1/24] Going Big on Mobile With Bo Ren: Lessons for Product Managers

Email subject lines: Going Big with Bo creative

Going Big with Bo creative

Yep, not much creative juices stirring here, but it almost always wins out in a four-way A/B/C/D test vs. “more interesting” or “playful” approaches, emojis included. That’s why we always seed our tests with at least one tried-and-true approach:

Going Big with Bo results

Overall, after three waves, we enjoyed an overall 27.3 percent average open rate when all was said and done — and a record number of registrations to that point. Was it the subject line, featured presenter, topic, audience (customers, engaged prospects) or all of the above?

Twist: Sometimes what wins an A/B open rate test may actually lose when clicks are counted. For example, my favorite option above, “Bo Knows,” won the subject line test for opens, but lost out to the basic “[Webinar, 1/24]…” in total clicks.

Mini-lessons:

  1. More opens does not necessarily guarantee success — be sure to frame any metrics conversation with what truly resonated, e.g., clicks and registrations.
  2. Always include a straightforward, “just the facts” approach — chances are it will come out on top more times than not.
  3. To lend a helping hand to your subject lines, start with engaging, provocative titles for your webinars, reports, events — and see if that curb appeal does the heavy lifting for opens.

Lesson 3: Keep the End Goal in Mind When Testing Email Subject Lines

For our spring webinar on preparing for EU General Data Protection Regulation (GDPR), we saw slightly below average open rates across three sends to customers and engaged prospects.

Email subject lines: GDPR webinar creative

GDPR webinar creative

Here are the results for the first wave:

GDPR webinar results

Sure, consumer privacy regulation is not the sexiest topic, but it’s one of the most crucial for marketers doing business in the EU. Yet — despite the lower-than-average open rates — we had the highest webinar registrations of 2018. (BTW: We feature ~12 webinars per year.)

So, we took it in the stride: only OK open rates based on testing, but more bodies in the seats. And the right bodies, as it turned out.

Mini-lessons:

  1. Keep your eye on the big picture: conversions, registrations, measures of success — rather than getting too focused on simple open rates.
  2. Consider the context of your send — a serious topic may call for a serious or straightforward subject line, while a playful topic allows for a similar tone.

Lesson 4: Remember “Pizza Is Life,” But Not Always for Email Subject Lines

To tie-in with March 14, International Pi Day — 3.14 on your calendar this side of the Atlantic — we launched a pizza bonus tie-in for engaged prospects: “Sign up for a demo, get free pizza!”

Email subject lines: Pi Day pizza creative

Pi Day pizza creative

For subject lines, we threw out all the stops: Puns, emojis, “the plain,” even a mathematical formula. In the end, no matter how you sliced it, the audience shrugged, “No thanks.”

Here’s one of our tests:

Pi Day pizza results

We had to stomach lower-than-average open rates — as much as 50 percent worse compared with prior campaigns to the same audience. As Yogi Berra once famously malaproped, “If people don’t want to come, nothing will stop them.” Worse, the campaign failed to generate any momentum.

Were we too clever for our own good? Pi Day not as universal as we thought? In the end, the crowd just wasn’t hungry for a demo to go.

Mini-lessons:

  1. Test your offer-of-the month to a smaller sample size before going big.
  2. Don’t fall in love with your clever self and subject lines — the crowd has a way of humbling you.
  3. Don’t be afraid to re-tool your campaign if initial tests show anemic results, because no amount of subsequent subject line testing will save the day.

A Final Word

I could hammer home sage advice like “test and test,” but I’d rather suggest the following when it comes to subject line testing:

  1. Don’t fall (too much) in love with your subject lines.
  2. Open rates aren’t everything.
  3. Don’t be reluctant to go basic in your approach.
  4. If they don’t want to open, you can’t stop ‘em.

Oh, yeah, find your Bo.

Leanplum has built email for the mobile era. We help mobile teams overcome common challenges, like siloed data sources and multiple vendors, to understand how email impacts downstream behaviors and conversions. All this makes good old email the ideal channel to reach and engage app users who opt-out from push notifications, abandon items in their mobile carts, or go dormant.