• by Loren McDonald , Op-Ed Contributor, December 17, 2009

The most frequently asked question from email marketers is some variation of this: "What is the average (blank) rate?" Ugh. 

I understand why marketers ask this and similar questions, but I have to ask: What are you going to do with the answer? If I told you that the average email open rate was 24%, how would you act on this data?

Don't get me wrong. I love benchmark studies as much as the next person, and I am finalizing one as I write this column. Benchmark studies and data have value, but I find that many marketers use the data incorrectly.

The Fine Print of Benchmark Data

When using benchmark studies for comparison, here are several points to keep in mind:

Average Comparisons: Most benchmarks are reported as averages, which isn't the most illuminating statistic. The median, which is the data point at the exact midpoint of results, is actually a more precise figure, but it's not reported in most benchmarking study results.

Value of "Average": I don't know about you, but I wouldn't expect my boss to get overly excited when I tell him that our marketing program was performing at a "C" level. In today's hypercompetitive marketplace, "average" has become the new bottom.

Average Calculation Bias: Averages are calculated by adding up a set of data and dividing by the number of data items. If the data set has a large number of outliers, the average can be skewed high or low.

Provider Bias: "Average" statistics are NOT industry averages. They are simply averages that are usually based on a subset of a service provider's client base or selected survey pool.

Process Variables: Do you use a double opt-in process, mail your subscribers on a daily basis, or utilize mostly trigger-based emails? These types of variables can mean huge differences in performance rates compared with published averages.

Peer/Purpose: If you are a B2B company using email primarily for lead nurturing, comparing your email stats to a study comprised mostly of B2C companies or companies using email to promote ecommerce is pretty meaningless. You have to become more discerning, to read the details of the study carefully to be sure it compares to your vertical, types of emails and other factors.

Rate Calculations: ESPs calculate rates in multiple ways. Example 1: Some include text emails or emails with images blocked that have a click as an open. Others include ONLY messages with a measured open, meaning that a tracking image loaded from the server. Example 2: Some ESPs calculate click-through rates based on clicks as a percentage of messages delivered, while others use emails opened as the denominator.

Using Benchmark Data Correctly

Now that I've established some of the potential weaknesses of benchmark studies, here are several ideas to help you use them correctly:

"In the Ballpark": The greatest value of average-based benchmark studies is to assure you that your program is in the ballpark. Your executives typically want this information, too, especially when it involves your budget allocation.

Forecasts: Benchmarks also provide a reality check. If your email revenue model assumes a 15% click-through rate from your broadcast emails, but the "average" is in the 5% neighborhood, your model likely needs to be redone.

Top Performers: Instead of comparing yourself to the average, or even the median, look for studies that break out metrics for the top performers and focus your attention on them.

Beyond the Basics: Look beyond the standard benchmark set (opens, clicks, conversions) to the measurements that are most relevant to your own business goals. These benchmarks can help you more precisely measure the direction your email program is moving and map out strategies to drive increased performance.

Peer Groups: While generally not available, consider forming smaller benchmark groups where you can better control the variables. If you lack peer benchmarks, use your program's past performance to establish benchmarks that allow you to look for trends and patterns.

"Are we meeting our business goals?"

In the end, the only benchmarks that matter are your business goals. Did your email program achieve the target goals that you and management set?

This is where your benchmarking energies should be spent. Having a 40% open rate on your last campaign that beats an industry average by 15%, or even a top-performers' benchmark by 5%, is irrelevant if you miss your company's fourth-quarter revenue goal.

Have any great benchmarking war or success stories? Share them in the Comments section.

Until next time, take it up a notch!