Just because you want to survey members doesn't mean you should. Uncover the best methods for getting answers.
Thousands of association members must curse the inventors of SurveyMonkey daily. The application's simple, user-friendly interface has made it easy for anyone with a question and an email list to create and distribute online surveys. But simpler is not always better when it comes to primary research. And just because you can doesn't mean you should.
Associations want to know lots of things from their members: Did you receive what we sent you? How did we do? Are you happy? Why haven't you renewed? Many other variations of the same questions are designed to yield one big answer: How are we doing at our jobs and at helping you do yours better?
Among organizations without trained researchers, the responsibility for surveying can be dispersed—or nonexistent. "Several different departments were surveying our members about a variety of topics all year long," says Kim Howard, editor-in-chief of ACC Docket, published by the Association for Corporate Counsel. "But there was no one on staff with real survey expertise. We thought that perhaps it would be best to consolidate these efforts, so that when we approach our members asking for information, we don't cause survey fatigue."
If your association doesn't have trained researchers in house, outside expertise can be helpful. "Many clients feel a real sense of relief around hiring professionals and getting their surveys done," says Kevin Whorton, principal of Whorton Marketing and Research. "We are a means to an end; they can check off that box."
The "end" Whorton refers to is the Holy Grail of primary research: a solid response rate. In designing surveys, Whorton aspires to a 20 percent response rate; anything lower signifies a lack of belief in the survey instrument and is not statistically significant, he says. Consultants can help with the major elements that drive a good response rate: timing, question design, segmentation, and reminders. They can also help with crunching the data produced by the survey.
The American Society of Consultant Pharmacists launched its first major membership survey in spring 2010. "We planned it for a time when members were registering for our annual conference, so we were top of mind," says Lance Clark, chief membership and marketing officer.
Determining who should receive a survey is almost as important as deciding what to ask, says Whorton. "It's best not to aggravate people for whom the survey is inappropriate. Don't survey all members about all of your conferences; be sure to survey only those who attended."
Before you begin, ensure that stakeholders are committed to using results to solve business problems. The California Society of Association Executives issues three major surveys on an annual rotation, asking questions about member needs, professional development, and communication. "The results help us understand what we are trying to accomplish. They drive a data-driven, decision-making culture that we have adopted intentionally as a response to the [book] 7 Measures of Success," says President and CEO Jim Anderson, CAE.
One word of caution: While surveys can be a vital tool for feedback, it's important to remember they are only one source, says Ross Simons, principal at management consulting firm Simons and Associates. "They shouldn't take the place of staff engaging members in multiple forums to generate information. Use survey results to conduct more informed conversations," he says.
Methods of Primary Research
Associations can conduct several forms of primary research to get the answers they seek.
Surveys. Ross Simons, principal at management consulting firm Simons and Associates, says that there are three key requirements for a survey. First, they should be designed with a specific business objective in mind. Second, their questions need to be as precise as possible and yield specific information. Lastly, they must be guided by an understanding of survey-research methodology. "A good survey will give you baseline information, but underlying questions may emerge," says Simons. That may create an occasion to move on to other research forms that allow you to discuss issues with respondents in more depth, such as focus groups and interviews.
Focus groups. Focus groups are another form of primary research, but their nature gives you less control. Surveys can answer the question, "What?" Focus groups and interviews are better at answering, "Why?" For example, when Lance Clark, chief membership and marketing officer for the American Society of Consultant Pharmacists, wanted to know what members thought of a redesign of its magazine, The Consultant Pharmacist, he convened focus groups.
Interviews. Like focus groups, personal interviews also constitute qualitative research. Kevin Whorton, principal of Whorton Marketing and Research, says interviews are underutilized. "In an interview, you can find out what people are familiar with, what they use, and how they feel about it," he says. "You can generate an interplay that starts from who the respondent is, and [it] allows you to ask questions in context and with nuance."
Testing. In many fields and in consumer direct marketing, testing is key. Rather than asking people what they think, split your membership and test your concepts for a new conference campaign or membership promotion. Testing is important, says Whorton, because what people say and what they do are often two different things.
Jennifer J. Salopek is a freelance writer based in McLean, Virginia. Email: [email protected]