Use A/B Testing to Learn What Brings Members Back

Membership Memo Associations Now May/June 2018 Issue By: Tim Ebner

The American Art Therapy Association applied A/B testing, also known as spilt testing, to a rudimentary membership question: Do lapsed members rejoin for instant gratification or long-term benefits?

When the American Art Therapy Association wanted to find the most effective way to bring lapsed members back into the fold, its membership team turned to a tried-and-true marketing and communications tactic: A/B testing.

Also known as split testing, A/B testing determines a winner between two competing messages or design concepts. By dividing an audience into segments and testing two different messages, marketers can see which communication received the most clicks, opens, likes, or shares.

This year, AATA applied A/B testing to a rudimentary membership question: Do lapsed members rejoin for instant gratification or long-term benefits?

Kat Michel, AATA’s membership manager, in consultation with Rita Santelli, president and CEO of the Savvy Org, LLC, tested this question with two subsets of former members who lapsed more than six months ago. Cohort A was offered a $10 Starbucks gift card for renewing. Cohort B was given a three-month extension on their annual membership.

“The good news was that our members seemed to respond more to the long-term benefits of an extended membership offer,” Michel says.

We were really focused on trying to get those lapsed members who have, for whatever reason, decided membership wasn’t for them.
—Rita Santelli

The difference was clear: Cohort B was twice as likely to renew as Cohort A.

That finding now informs AATA’s win-back strategy, which emphasizes new membership benefits like a recently launched online member community.

“We were really focused on trying to get those lapsed members who have, for whatever reason, decided that maybe membership wasn’t for them,” Santelli says.

Split testing sounds simple, but it takes time to set up. Before you try it, check whether your email marketing and association management systems can handle a two-prong test.

“Right now, our AMS doesn’t allow for us to use discount codes or coupon codes, which was a major drawback and added to the staff time for this project,” Santelli says.

That’s not to say you shouldn’t give A/B testing a try. You can use manual configurations, like email segmentation and suppression, but Michel recommends looking to technology to help automate the process.

“Don’t be afraid,” she says. “Just be ready for a few extra turns if you don’t have the right tools in place.”

[This article was originally published in the Associations Now print edition, titled “Membership Memo: Sweeter Deal."]

Tim Ebner

Tim Ebner is communications director and press secretary at the American Forest & Paper Association in Washington, DC. He is a member of ASAE’s Communication Professionals Advisory Council and a former Associations Now senior editor.