How to Measure International Efforts

By: Michael Michaud, CAE

The American Society of Mechanical Engineers' commitment to growing internationally involves more than translations and policy statements. Here's how ASME crunches its numbers to discover what does and doesn't work around the world, down to the last web visitor. (Titled "The Measure of Global Success" in the print edition.)

It's hard to read the news today and not be reminded of how the earth is shrinking. Disasters, both natural and manmade, seem to occur more frequently, and the companies that employ our associations' members face increasingly conflicting dynamics. To grow and remain competitive, those companies need to seek new markets and expand their supply chains. If they are not already active internationally, they are moving in that direction, either building fences to isolate themselves or bridges to connect with others. All this can create tensions in any organization, but these changes can also create opportunities for those who are quick to react to or anticipate the shifts.

It seems clear now that globalization isn't the next challenge but rather the context in which we need to consider all future challenges. The search for clean energy, potable water, and healthy, famine-free communities are just some of the issues that require global thinking. The solutions to these grand challenges may lie in the ability of associations to leverage their knowledge, engage their members, and call their communities into action on a scale never before seen.

Becoming a global association can be daunting, but it can also be strengthening: The association can build new markets, attract members, and increase its relevance. The decision to become globally relevant, to serve a global community, or to actually become a global organization are all strategic decisions. Once that decision is made, though, your organization will need new metrics to measure its progress.

The American Society of Mechanical Engineers, a 501(c)(3) headquartered in New York with offices in Belgium, China, and India, has been active globally for decades in terms of members, conferences, and publications, and our technical standards are now referenced or used in 120 countries. But though ASME already had a significant international presence, in September 2008, ASME's board of governors decided to reaffirm a strategic focus and commitment to globalization. While the direction from the board was clear, it was also very general and open to interpretation: Where did we start? What did this decision mean exactly? How do we make it happen? Did the board want us to focus on major markets or "be" everywhere? Could we afford to be everywhere? If we only focused on our major markets, were we true to our mission and the board's direction of being global? Each question seemed to lead to another.

To tackle these issues, we established an eight-person Global-Strategy Execution Team (known internally as the G-SET), chaired by me and including staff members chosen for their international experience, both inside and outside of ASME, as well as their knowledge of its major business units. At our first meeting in February 2009 we determined that becoming a more global organization meant focusing on global growth. We needed to increase our presence in new markets, but we also needed to increase our impact in markets where we already had a presence. The more the G-SET looked at it, the more it became clear that the board's direction was about having a greater, more global impact. We refocused the initiative around increasing ASME's global impact, which resonated throughout the organization and translated to the staff's personal objectives.

"Impact" is about relevance; "global" is about scale. Both are difficult to measure with any accuracy. Of course, we could easily have committed the organization to developing three new relevant programs per year, signed a new partnership agreement every six months, and declared our global initiative a success. Somehow, this approach seemed a bit shortsighted. We needed a measure we could use over an extended period of time to see just how much progress we were making.

At the G-SET's second meeting a month later, we began to close in on the idea of developing an index that would measure the key drivers of ASME's activity, collect them on a country-by-country basis, and then compile them to produce an index, much like the stock market. Increased activity would represent greater growth. If we were shrinking or if the rate of growth was lower than expected in certain areas, we could adjust our plans accordingly. 

Building the Index

When designing the index, we made two general assumptions about how we would have a positive global impact: by having relevant products, programs, and services and by marketing and distributing them globally. We identified several indicators we knew we could individually measure accurately (on a country-by-country basis) and collectively could represent ASME's overall impact. The indicators we chose were

  • Web visits. With new tracking software, the number of web visits gives us an accurate indication of current use and potential future business. This is the closest thing to a leading indicator we could accurately identify and track with internal data.
  • Membership. An important measure of an individual's personal engagement with or commitment to the organization.
  • Participants. We gathered numbers from both our conference and training activities. Each is a separate and distinct core competency, and these measures track end-user commitment to ASME as a valued provider.
  • Volunteers. Particularly important to an organization that develops consensus-based codes and standards, volunteers represent a level of commitment to the organization and the products developed. Without volunteers, we would not have content. Without volunteers from around the world, we would not have globally relevant content.
  • Accredited manufacturers. These are the power users of ASME's codes and standards. These companies commit to a third-party inspection of their use and application of various ASME codes. While not encompassing all of ASME's products, these companies employ members, encourage training and participation in conferences, and support in-house experts on code committees.

One thing we chose not to include in the index: sales revenue. Because many of ASME's products are sold through agents, corporate-buying offices, or third-party consolidators and shipped around the world, we don't always know where our products are being used. In addition, if we only tracked revenue, the numbers would be misleading: Some products, such as those related to membership and training, are adjusted for local purchasing power while others, like codes, standards, and other publications, are not.

Each of the indicators we collected reflected a voluntary act from an individual who visited, joined, attended, participated in, or paid for something from ASME. This transaction indicated that we were producing something of value and were having an impact. The degree of that impact, however, varies—there's a difference between the casual web visit and the long-term commitment that becoming an ASME Certified Manufacturer involves. So we developed a system of weights. We considered assigning each indicator a weight equal to its dollar value but quickly dismissed this because not all indicators had real dollar values.

Because we had data points from four quarterly periods, we were able to run some scenarios to see how an increase in a particular area would affect the trend lines. In the end, the G-SET agreed that we needed to use weights that felt right and would not cause major swings yet would reflect significant trends from any of the indicators in the overall results.

The result of the weighting is as follows: Certified manufacturers received 250 points; committee members received 50 points; members, course participants, and conference attendees were all given 10 points each; and web visitors received only .2 points, recognizing that their participation was casual but that web visits were an important leading indicator we nonetheless wanted to capture, especially when entering new markets.

Sample Index in Action

ASME uses the data in the chart at the below to see at a glance how successful its international efforts are in a particular country. The overall index for India increased 38.5 percent between the fourth quarter of fiscal year 2009 and the fourth quarter of 2010. The lead index figure (92,120) is calculated by adding the weighted numbers for each of the categories listed. ASME stresses the importance of its 160 certified manufacturers by multiplying that number by 250 for the index; the 39,650 web visits from that country are relatively deemphasized by multiplying the figure by 0.2. The second chart shows how India's index has risen in the past two years and how it relates to other regions ASME tracks.

INDIA

+38.5%
92,120
web visits membership
+1.8% +115%
39,650 3,852
training-course attendees conference attendees
+87.1% -61.6%
159 203
certified manufacturers S&C committee members
+15.9% +925%
160 41
Percentage reflects growth Q4, FY09 to Q4, FY-10. Value reflects current (quarterly) volume


Click chart to enlarge.

Understanding the Data

We were now able to provide the board with an indicator of progress toward our goal of having a greater global impact. We track both the actual volume of improvement (important for comparison between markets and countries) and the percentage of improvement over the same cycle in the previous fiscal year. In the long term, this will create a trend line, similar to a mutual fund tracking performance over different time periods. While this perspective or view was important for the board to help shape investment and deployment decisions, we were also able to develop "sub-views" for each indicator, which have helped operating units adjust tactics. With only one year's data, the index has already triggered adjustments in country-specific strategies in and between product departments.

As individual units provide data and see the impact they have on the overall outcome, the index becomes a powerful management tool, particularly for organizations that use performance-metric-based systems. The data provided by the index will help any society as it struggles to manage its resources to serve the world's marketplace. The index has clearly identified growth opportunity areas and has reinforced decisions to develop products that address specific local requirements in terms of price, language, and other factors.

The ability to incorporate other data, like sales figures, brings an additional dimension to the index and can help reduce organizational risk with more accurate forecasting and better predictive modeling. While we resisted the temptation at the outset to incorporate external data into the index, the possibility exists in the future to compare internal trend lines with outside data such as GDP, education levels, power consumption, and even greenhouse gas emissions. While we're not quite there yet, this could ultimately be the test of an organization not just being "used and therefore useful" but having a significant global impact.

The index provides perspective that helps ASME meet its mission and grow globally. It helps ASME find the balance between serving the world as a whole and focusing on specific markets. As we continue to collect more data, we will not only have a better understanding of "place"—where we are and where we need to go—but we can also better manage "pace" through budget and resource allocation.

Strategic leadership, strong planning, and execution, when reinforced by real data, can only help make the organization stronger. In our case it is helping us become more self-aware and more attuned to the global environment at the same time. All this helps ASME position itself to tackle the next challenges that lie just over the horizon. 

Michael Michaud, CAE, is managing director, global alliances, at the American Society of Mechanical Engineers in New York City. Email: [email protected]

Sidebar: Building Your Own Globalization Index

Thinking of creating globalization metrics of your own? Here are eight steps for associations looking to create an index similar to ASME's:

  1. Identify key indicators. These should relate to the foundation of the organization's mission and vision.
  2. Resist the temptation to count everything. You're looking for information that's both relevant and easy to collect on a regular basis.
  3. Assemble data over time. Ultimately, you're looking to see how trends play out in the long term, so you can adjust your international strategy accordingly.
  4. Run scenarios. If your index gives you results that don't seem right, you may need to adjust your weighting.
  5. Get senior approval. The index you arrive at won't be perfect, but leadership should know that trends never are and that real value lies in long-term use.
  6. Incorporate your findings into a governance dashboard. You'll want to give your board a quick view of what you're finding.
  7. Develop both "stock" and "mutual fund" views. You'll be looking at individual countries, but also consider larger "baskets" of countries as you consider regional approaches.
  8. Create operational views for staff and departmental perspectives. Let each department know how its efforts are working (or not) in different countries so the department can do its own research and make adjustments.

Online Extra Video: Global Initiatives

Michael Michaud, CAE, managing director, global alliances for American Society of Mechanical Engineers describes a tool ASME uses to make strategic decisions about its global expansion.