Measuring Non-Financial Returns: Putting the Cart before the Horse

Multiple-bottom line (MBL) investing – call it #impinv, #venturephilanthropy, #socialcapital or (for exchange-traded instruments) SRI – by definition requires its practitioners to generate and report on performance across financial, social and environmental parameters. These last two categories have been difficult to pin down. For those coming from the development finance vector, establishing causality between actions and outcomes is considered necessary and desirable. On the flip side, actors coming from the mainstream finance vector (VC/PE, investment banking) want to establish correlations that provide “market intelligence”. Both groups want the ability to explore comparable data and determine benchmarks so that investment managers have feedback loops and investors can make informed decisions. In the absence of historical performance data and benchmarks, money will flow to charisma and connections first, with substance and operational excellence playing a limited, later role.

Make no mistake, we need comparable data on non-financial returns if MBL investing is to succeed. One problem I see arises from an irrational exuberance for determining causality around all desirable social outcomes before a sufficient number of mature MBL investments are available for analysis. Is one business model better than other? How will we know unless we have examples of mature companies using different models? In parallel, I worry about the tension between wanting to understand how we can produce the best social outcomes and needing to be respectful of the dignity and privacy of clients. Causality can best be approached through long-term data and analysis, but correlation/market intelligence requires short-term data points – and both types of indicators work better if baseline knowledge is collected at launch, for comparison to outcomes achieved down the line.

Efforts to build an #impinv eco-system have rightly identified the need for comparable non-financial data, but the emphasis is placed on how to measure what we want to know, rather than on understanding what outcomes we are achieving in practice and how these were obtained. Let me try to illustrate my point. The Global Impact Investment Network (@GIIN) defined “comparable benchmarks” as a priority and set about creating IRIS to address this gap. Collaborative efforts also gave birth to PULSE software that can make it easier for firms to track @IRIS indicators over time. This is a great and important idea, but as soon as we moved from intellectual support to practice, we ran into three problems:

  1. A fair number of IRIS “indicators” are actually descriptors. For example, the number of Board members and frequency of Board meetings provide insight into governance, but these aren’t necessarily indicators that change over time. Descriptors are important for communicating to potential investors or portfolio companies, but they don’t provide a manager with analytical info or market intelligence;
  2. Multiple indicators for each sector theoretically enable managers to make a strategy-driven choice without losing comparability. Unfortunately, the metrics around some of these indicators will be impossible to obtain in emerging markets (and even in some areas within “developed” markets). Determining renewable versus non-renewable energy use is clearly desirable, but can the local utility provide this data? Will I be able to use this indicator only with portfolio companies that generate their own renewable energy? In some jurisdictions you can contract for renewable energy from your utility, in most, not so much;
  3. The management variable lacks performance metrics. I understand that much of what defines “excellent management” is intangible and non-quantitative. Nevertheless, given my stated goal of empowering and enhancing quality of life at the BOP, there is data around VSI (ventures with social impact) management that is worth understanding. Level of education is one such variable. Are we financing entrepreneurs with a high school education, or the scions of oligarchs with advanced degrees from name brand universities? I’d be interested to know if success rates differ across these two groups. Years of direct and indirect experience is also a potentially relevant indicator. Is it enough to have passion and vision, or are there specific skill sets that VSI managers should bring on board sooner rather than later? Knowing something about VSI management compensation structure is worthwhile too. Is base pay above, below or at market? How is management rewarded for short-term performance? For long-term performance? Are all employees included in such incentive schemes, or only managers? How do these factors align interests and contribute to the sustainability of the enterprise?

The issues I’ve described above can all be overcome in time, but one additional concern remains. Technology can get us better and more frequent access to self-reported data. Can we adapt tools like Frontline SMS to develop more useful analytics around correlation? Is it more important to deliver self-reported data that may tell us how to improve the goods and services on offer, or to understand how these goods and services change the lives of consumers? Is it possible to do both using the same data set? One challenge is to try to define every social outcome data that we can collect information about today, at whatever cost. It is quite a another challenge entirely to look for the best and most cost-effective indicators that tell us where we are today and offer insights for better outcomes tomorrow.

Last, but by no means least, performance measurement looks at outcomes, but overlooks the question of how performance is attained. This is a “money in the middle” issue – what does it take to generate top-tier returns on MBL investments? Which factors and actions are most meaningful when it comes to improving or “adding value” to investments? Is a robust governance program worth the cost? Can better technology facilitate timely and accurate reporting? How much staff talent should focus on origination, versus structuring and value-add? What are the parameters that define “operational excellence” in MBL investing? How we achieve great performance should be as much of a focus as how we measure performance. After all, if we don’t produce consistent and desirable MBL returns, performance measurement will be our death knell rather than a key component of industry growth.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s