The Metrics of Engagement

By: Stephen G. Pelletier

Trying to make metric sense of your association's social networks? The answer transcends mere statistics.

Picture this: The XYZ Association has plunged into social media in a big way. Behind a firewall, its members-only communities buzz with discussions of pressing issues. Similar topics get intense scrutiny in the association's public community. XYZ's blog hosts a rich exchange of posts and comments. XYZers frequently tweet on Twitter, friend on Facebook, and network through LinkedIn.

Up to her eyeballs in hashtags, posts, and pokes, the association's young social media manager, Karen, watches all this traffic intently. She knows it was no small victory to get XYZA to make a bold foray into social media. But mindful that the association likes to make decisions based on metrics, she knows she needs to find ways to analyze all the traffic in ways that will make sense to managers who like plain numbers in black and white.

Not long ago, when websites were the only social media, associations measured effectiveness in terms of hits. Eventually, we learned that individual page views are a better measure. But today, that's no longer enough. For many associations, social media and mobile devices have become interlinked with websites. Accordingly, that change requires more nuanced ways of understanding the impact social media is having on members. We need a richer vocabulary of metrics and better tools for measuring results.

That understanding has given birth to a new perspective on social media. The new vocabulary that has evolved might be called the metrics of engagement.

Ready, Fire, Aim

Chris Bonney is vice president of client experience at Vanguard Technology, which helps associations improve their online and social networking operations. Over the last several years, he has seen many associations jump into new facets of social media without a strategy beyond "we have to get into this or else get left in the dust."

On one level, Bonney thinks there's value in that "ready, fire, aim" approach: It gets associations into the social media arena. But once the initial excitement settles, Bonney says, associations need to pause to better understand why they got into social media and what gains it might bring. What drives that need is part marketing and part demands for quantifiable results from association board rooms, C-suites, and other critical stakeholders.

For example, Bonney cites the experience of one of his association clients, the Chicago-based International Special Events Society. ISES plunged aggressively into social media several years ago, in part to give it a competitive edge over other organizations in its industry. It has a prominent presence on Facebook and also maintains members-only communities using software from Higher Logic, a provider of social-network platforms that works with approximately 250 associations.

Bonney says staff at ISES soon started wondering, "Now that we have this traction, what does it mean? How can we measure it? How can we act on that measurement?" To answer those questions, Vanguard took a close look at the association's online presence to profile what Bonney calls the "state of the community."

One huge takeaway was that top users of the association's online communities were members whose names neither staff nor volunteer leadership recognized. That showed the association's management that social media could lead formerly nonactive members to become more engaged. A related lesson, Bonney says, is that "people want to interact with our association in their own way." Some prefer Facebook, some like Twitter, and still others like the familiar format of listservers.

Using those findings, the association defined one new metric for success in its use of social media as "engaging people who have never had a way to engage with us that works for them." How many people need to do that before an association can claim success? Bonney says it's up to each organization to decide the number that's right for them.

Bonney estimates that some 30 percent of associations cling to the now-outmoded approach of simply measuring traffic to their website as a key metric. Another 40 percent, he thinks, are even further behind the online-presence curve and may not be measuring much of anything. (In general, both vendors and association staff say, associations lag the corporate world in creating social media channels, much less fully exploiting them for business gains.)

In contrast, Bonney says, savvy associations are now looking in depth at what visitors do when they visit association websites and social media outlets. It's all about conversion, he says. "Did we get a new membership or membership renewal? Did we get an event registration? Those are the types of metrics we should be looking at.

"What actions are we trying to get users to take on our site, and what do those metrics indicate? That's really the next level," he says.

Bonney says software like Google Analytics is now robust enough to give association managers practical insights about the path visitors to an association's website take. For example, if metrics show that a particular page isn't leading to desired actions—such as a meeting registration or a publication purchase—web managers can move quickly to change the pathway to guide users to pages that have proven more successful. Association staff, including ISES Operations Manager Kristin K. Prine, also like the metrics that are built into Facebook, such as information on page views, external referrers, and active users.

Member Engagement

Bonney says a small but growing group of associations are going even further. The most sophisticated users of metrics are analyzing the data they collect to assess what they say about member engagement in the association's work. Metrics that point to engagement might identify all the top contributors to online discussions, how many times they log in, and what they do while they are online. Associations can now even drill down to a level that tells how many times a user has a friend request accepted or a post forwarded. Metric tools have become so sophisticated that they enable such information to be tabulated algorithmically, pooling data in ways that produce deeper insights into the use and impact of social media.

Heather McNair has experience with social media in two associations. Formerly on staff at the Medical Group Management Association, she is currently vice president of marketing and social media at the American Association of Nurse Assessment Coordinators (AANAC). At MGMA, she says, metrics for social media were integrated into an associationwide effort to measure accomplishments beyond the financial bottom line using a "balanced scorecard," a group of metrics shared with MGMA senior staff and board members. One portion of MGMA's scorecard was devoted to membership engagement, she says, with a particular focus on member communities behind the association's firewall.

"We looked at things like how many people updated their profile, how many unique subscribers there were in our discussion groups, and how many people downloaded resources from the libraries where members share documents," she says. MGMA CFO Leah Brash notes that member engagement includes community postings, e-newsletter open rates, survey participation, and other metrics in addition to website visits. "Just visiting isn't true engagement," she says.

"At MGMA, we used data to inform our next business moves," says McNair. "We recognized that the more engaged members are, the more likely they are to renew. We used that premise to fuel the business process, looking at the cycle of getting more people involved." Another benefit? Putting the results of those metrics on the balanced scorecard raised awareness across the association as a whole about the importance of social media.

At AANAC, McNair's challenge is to integrate the association's relatively young social media presence into a new IT infrastructure. Like ISES, AANAC relies on the metrics tools in Higher Logic software. Higher Logic Chief Operating Officer Andy Steggles says the software helps AANAC measure member engagement by assigning point values to each member's online interaction. Looking beyond members who merely browse online, extra points go to "contributory" actions, such as posting to a community, versus noncontributory, which might mean simply reading a posting.

"The association assigns point values based on different levels of engagement and defines point values based on what they're trying to achieve," Steggles says. The system enables the association to "measure levels of engagement and activity down to the individual level, based on the weight that is assigned to various activities." (For more on this system, see "The Path to the Best Database" on page 10.)

Looking at social media broadly, consultant Dan Blank says the most important metric of all may be its ability to be action oriented. The founder of the online media consulting and training company We Grow Media, Blank says he prefers discussions of metrics that focus not on spreadsheets but on missions and how an organization is "trying to execute on that mission and engaging in understanding their audience." Moreover, he argues, the goal of looking at data about the use of social media should be to find an insight that can be acted on.

"Metrics are useful when they are actionable—something that leads to real business decisions," he says. "It's not about living in the data but living by the data."

Sidebar: Advice From the Field

Practitioners who have been in the social media trenches for a while have plenty of advice for associations that want to improve their use of social media.

  • Map social media to your organization's strategy. Chris Bonney, vice president of client experience at Vanguard Technology, urges association staff to step back from the daily whirlwind that social media creates to analyze it in the context of your organization's goals. "Don't treat technology as a silo and try to do things that don't complement and map to what you are trying to do as an organization," he says. Andy Steggles, chief operating officer of Higher Logic, adds that associations should "think about every department of your organization and how they might be leveraged through social media."
  • Focus on the functions. Ahead of software, one key to success in social media is helping your members understand the kind of interchange they want and value with each other, then creating the right platform to support that. Steggles finds the number-one mistake associations make is simply announcing that new social media tools are available. Instead, he says, focus on the functions the platforms can perform. "Tell members that you've upgraded the membership directory to be more interactive and dynamic," he says. "Tell them that you have a new online library where they can share resources directly. Show them how your new discussion group will permit a much deeper level of collaboration.
  • Content is king.Bonney says each facet of social media needs to have its own content. It's not enough to post to a blog, then tweet about that post and forward it to Facebook. Associations should have the depth to publish different content on different social media—posts that each provide "a unique offering to your membership," he says.
  • You're not just a website anymore.While a website remains a vital portal to an organization and its social media, associations need to consider their broader roles as content distributors. By changing the focus in that way, Bonney says, the key question becomes what channel works best to distribute different content.
  • If you build it, they will … need incentives. It's not enough to simply make social media available; you have to also get people active in it. Offer incentives that will entice members to populate, use, and champion your social media. "Let them attend some of the demos as you're vetting software vendors," says Bonney. "Ask their opinions and involve them in key decisions around your social media."
  • Believe in it.Advocates within associations need to strongly believe that social media is a vital tool, while recognizing that some might be ambivalent or even hostile toward it. "Believe in it, and dedicate resources to it," says Heather McNair, vice president of marketing and social media at the American Association of Nurse Assessment Coordinators. As she helped implement social media at AANAC, McNair says "we had no idea how much time and dedication it was going to take. But it's totally worth it." She credits social media with playing a large role in AANAC's success in retaining and adding members during the recent recession. "I think a lot of it was because we were committed to engagement with our members," she says.

Stephen G. Pelletier is a writer and editor based near Washington, DC. Email: [email protected]

Stephen G. Pelletier