Member Surveys and Beyond: What to Do With All That Data

Membership Data February 1, 2016 By: Joyce O'Brien

Association membership and marketing professionals have more power to collect and analyze data than ever before, but success still relies on sound data strategy and survey tactics.

Associations typically produce reports on their members regularly, tallying member counts, calculating revenue, counting how many have renewed and how many are new. They also conduct member and customer surveys. What happens to all that data? How is it analyzed and used?

Too often the magnitude of data can overwhelm even the most seasoned membership professional. How can this critical tool be used in understanding, engaging and growing your membership?

Joe Colangelo, founder and CEO of Bear Analytics, a firm specializing in association data analysis, suggests a three-step process to jumpstart your member analysis process.

  • Create your association's data map. Understand where your data assets come from and who in the organization controls that data. "Going through this process is an exercise in transparency and really sets the stage for comprehensive data analysis," says Colangelo.
  • Identify the metrics that matter. What information do you collect about members and customers, and what does it tell you? "It's not always the largest segment of your membership that is driving growth," says Colangelo. "Look at some of the smaller segments of your membership to identify where your growth potential is."
  • Look for patterns. Isolate transactional data (attendees at your annual meeting, for example) and analyze it. Who comes to your association's annual meeting? How often do they attend and at what frequency—every year, or every other year? How does annual meeting attendance impact membership? What organizations send the most employees to your event? By analyzing data to answer these questions, you can begin to impact membership marketing and sales.

A Case Study in Data Analysis: American Society of Plant Biologists

Like many associations, the American Society of Plant Biologists (ASPB) relies on subscriptions to its professional journals for a large percent of its annual revenues. The association needed to evaluate its traditional business plan in light of changes in government funding related to open access. With 4,000 members, the association also wanted to understand more about its nonmember community (readers, authors, event attendees, and so forth) to determine a new vision and strategy for the organization.

A survey measures what members think they do, but looking at the transactional data shows us what they actually do. Gina Scime, associate director of market research, American Pharmacists Association

Moving toward a robust digital platform to galvanize the plant science community and to connect people and content, ASPB needed to understand its options and potential as a member organization. "We knew our opportunity was in the size of the community surrounding us," says Susan Cato, director of digital strategy and member services. "We have a world of people interacting with us, but we weren't calling them members. We need to figure out how to make our organization more inclusive and build our brand, while at the same time helping to strengthen the plant science community. I needed a hard look at our data."

Combining 19 separate data sources and spreadsheets, Bear Analytics was able to take the most recent 10 years of data from ASPB and, after scrubbing it, identify 40,000 customers who had interacted with ASPB at least twice, such as attending meetings, submitting papers, or reading the journals. "Now that we know we have a community that is 40,000 strong, the next step is to evolve the membership model along with a set of products and tools to meet their needs," says Cato.

Survey Says …

Surveying members and customers plays an important role in interpreting existing data and offers a window into perceptions, satisfaction, and unmet needs of your primary audiences. "Survey data represents the respondents' perception," says Gina Scime, associate director of market research for the American Pharmacists Association (APhA). "Is what people think a true reflection of what they do? A survey measures what they think they do, but looking at the transactional data shows us what they actually do."

Scime offers her top five tips to get the information you need through surveys.

  • Match the recipients to your membership. Start the process with a list that is a demographic match to your membership. "Having a representative sample is one of the most important elements of a successful survey," says Scime.
  • Ask the same questions. Some questions should be on every member survey. "For organizations that don't have research departments, if you are doing one piece of research every year, the member satisfaction survey is the one study that every group should do," says Scime. "It's a scorecard of how you're doing; it's a measurement of the sum of everything you offer." She recommends three core questions: overall satisfaction, likelihood to renew, and likelihood to recommend membership to colleagues. By looking at the results over time, association staff can monitor member perception to see if satisfaction and other measures are moving in a positive direction.
  • Get specific. Measure familiarity and usage with your member benefits to assist in member recruitment and retention. What benefits are members and customers most interested in? What benefits are falling flat? Sometimes survey results point to a failure of marketing or communicating a specific benefit to members in a way that catches their attention and promotes usage.
  • Take a closer look. Use cross tabs to pinpoint perception, preference, and satisfaction in specific audiences. "Most survey software offers the ability to group the findings by key variables and audiences," says Scime. "It's to your advantage to look and see if there are significant differences in those groups." Looking at subsets of your members through the benefits that appeal most to them, coupled with the transactional data analysis of products members use, will help you create more compelling membership messages.
  • Think outside the traditional survey box. Scime recommends these two lower-cost but effective survey options:
  • Focus groups. Take advantage of opportunities when you have groups of members and customers together, such as at your association's annual meeting. For the cost of a meal, you can assemble a group of diverse members and nonmembers. "We've found that members are excited to participate in focus groups and are flattered that we are asking their opinions," says Irica Cheeks, APhA director of marketing. Conducting focus groups via telephone is relatively new and also effective. The only costs are in recording the call, paying a moderator, and the incentive to participants.
  • One-to-one outreach. Scheduling personal interviews via telephone between staff and select members is perhaps the lowest-cost option. Something as simple as a welcome call coupled with a few specific questions about why the member joined and what they hope to receive as a member can gain insights into what makes prospects join your association.

One thing that both Scime and Colangelo advocate for is increased visibility of the data collected within associations. "We have budget meetings, strategic planning meetings, but why don't we have data team meetings?" Colangelo asks. "We all have a role in data—from the creation, analysis, generation, and use. Data impacts all of us."

"Increasing the value of research to an organization seems to be a trend," says Scime. "Organizations inherently have their own biases. Even when looking at survey results, staff tends to see the research in terms of what they believe to be true about their members." She suggests making the results available across the organization to spark discussion and to test those internal biases.

"I was listening to a researcher tell a story about how the company sales executives believed that customers didn't buy products during a particular season. Research suggested something different," Scime says. When the company finally marketed products during the "slow" season, it realized an increase of 25 percent market share during those "slow" months.

With increases in technology and the ability to collect data, understanding how to evaluate the data you collect, both transactional and survey data, can play an important role in recruiting, engaging, and retaining members.

Joyce O'Brien

Joyce O'Brien is senior director for membership at the American Pharmacists Association in Washington, DC.