Test, Measure and Refine Your Way to Direct Marketing Success
Renowned advertiser Claude Hopkins wrote this about the importance of testing, measuring and refining in his book Scientific Advertising,
"Almost any question can be answered, cheaply, quickly, and finally, by a test campaign.
And that´s the way to answer them – not by arguments around a table.
Go to the court of last resort – the buyers of your product."
Test, Measure and Refine (TMR)
The process of testing, measuring and refining is what makes direct marketing different from advertising and other types sales and marketing disciplines. TMR is the crux of direct marketing.
Whether sophisticated or simplistic, TMR is a process of trial and error—a systematic process of trying different approaches to determine what works best.
Let´s break down the root elements of TMR:
Testing is a matter of trying out advertising or direct marketing on a small scale before attempting a similar effort on a larger scale. A typical test involves two or more variations of a single variable being tested against one another at the same time.
You can test a number of different things, including mailing lists, offers or product/service pricing. You can even test different creative approaches. For example, you can test a direct mail message in a standard #10 envelope vs. a similar message on a stand-alone 6"x9" postcard.
Note that testing something, no matter how small, is better than not testing at all. But it´s important to remember that tests should be valid, whenever possible.
Validity can become a very complex topic and is, by-and-large, outside the scope of this article. For the time being, just remember this rule of thumb: you should test two or more variations of only a single variable at the same time.
If you want to test mailing lists, for instance, you should send the same direct mail piece, in equal quantities, to 2 or more lists. The recipients should be randomly selected so the responses aren´t biased. To do this, ask your list provider to do an "nth-name selection", whereby they randomly choose every xth name (e.g. every 5th name) up to the desired quantity.
It´s very important to only test a single variable per mailing. Why? Every situation is different, whether due to timing or who received it. Once more than one variable is introduced into a test it becomes very difficult to draw clear conclusions.
You can contact me at barnold@jarboedirect.com or 812-322-3524 and I can help you with your test design—or I can refer you to someone else based on what you´re hoping to accomplish.
Measuring is the process of counting replies and evaluating results. By doing so, you can learn what worked and what didn´t work in your test.
There are many different ways to measure your results:
- Qualified responses—legitimate responses based on the criteria you´ve previously selected
- Conversions—setting an appointment with a direct marketing respondent
- Closures—selling a product or service to a direct marketing respondent
- CPR—cost per response
- CPO—cost per order
- CPA—cost per appointment
- CPS—cost per sale
The minimum CPO is also known as an "allowable". Your allowable is the point at which your revenue equals your costs. If you´re selling a product or service at or below this breakeven point, you´re losing money. Conversely, if you´re selling your product or service above the allowable, then you´re making money.
The CPO-allowable calculation should precede every test. You can also calculate an allowable for CPR, CPA and/or CPS. The amount you´re willing to pay for a response, appointment or sale should be based on the lifetime value of a customer or client. (We´ll discuss lifetime value in the next article.)
Once you set an allowable, you can track your responses and calculate your cost for each response, appointment or sale to see if it meets or exceeds that allowable. If it does, make adjustments to decrease your costs and/or increase response.
Note: Your CPO should exceed your CPO allowable, whereas your CPR should be lower than your CPR allowable.
If you´d like to try your hand at calculating a CPO breakeven point, send your email address or fax number to me at barnold@jarboedirect.com and I´ll send you a formula.
Refining is the process of looking at the results of your tests and making adjustments or enhancements based on those results. What and how you refine is based on the manner in which you measure.
Here are some thoughts on making adjustments:
| Situation: | Diagnostic ideas: |
| Low responses | Correct target/list source? Reconsider offer/price/position. |
| High response, low conversion | Relevant offer? Is your sales/telemarketing doing its job? |
| Low response, high conversion | Find other prospects that look/act like the new customers. |
| Over cost-per-sale allowable | Select a better offer, reduce costs and work with sales team. |
You can decrease your costs by choosing or negotiating a better media buy, or perhaps by producing more cost-effective direct marketing. To increase response, reevaluate your mailing list, offer and product/service price before considering a different format (e.g. #10 envelope vs. 6"x9" envelope).
Here´s a simplified example of how TMR works:
Step 1: Determine CPO allowable.
Step 2: Test 10,000 direct mail pieces—5,000 with price A, 5,000 with price B.
Step 3a: Measure price A.
Step 3b: Measure price B.
Step 4: Identify CPO and compare to CPO allowable for both prices.
Step 5: Identify best price and incorporate what you´ve learned (refine).
When you´ve tested on a small scale and are satisfied with the results, you can do the same test on a larger scale. Direct marketers call this a "roll out".
It´s all about getting a response
Because the target audience is the ultimate judge of your direct marketing by virtue of whether or not they respond to your message, you have to be able to test different approaches, measure the results and refine your direct marketing based on what you´ve learned to maximize responses.
The practice of testing, measuring and refining exists because direct marketers realize the ultimate judge of any direct marketing is the target audience. Keep the TMR process in mind before, during and after all of your direct marketing efforts.
In the next part of the series, I´ll talk about the importance direct marketers place on lifetime value. Until then, feel free to call or email me with your questions or remarks.
Brian Arnold can be reached via email or at http://www.jarboedirect.com.
Brian Arnold is a direct marketing consultant and the President of JarboeDirect. You can reach him at barnold@jarboedirect.com or toll free at 1-800-804-0830, if you have questions.