Social Return on Investment vs. Cost-Benefit Analysis
How different are they really? Here’s my view.
Social Return on Investment (SROI) and cost-benefit analysis (CBA) - what’s the difference? Both offer valuable principles and practices. However, if SROI and CBA were used by different people to evaluate the same investment or intervention, they could lead to different conclusions.
SROI and CBA share a common goal of understanding whether something creates more value than it consumes. Their similarities are more striking than their differences, though they do approach the analysis from slightly different angles: SROI emphasises social value and stakeholder perspectives, while CBA prioritises precision and reliability. CBA has a longer history and more established professional community, while SROI has gained traction by making assessment of costs and benefits more accessible to non-economists. Both methods have their challenges, with SROI sometimes lacking rigour and CBA overlooking stakeholder engagement.
To bridge these gaps and offer more comprehensive evaluations of value created and consumed, I argue that how well the methods are applied matters more than which one is selected. And whichever method is used, I recommend including them within a broader evaluation framework that incorporates evaluative reasoning, mixed methods and stakeholder participation. By doing so, we can unlock the unique and powerful insights offered by CBA and SROI while addressing their limitations.
SROI and CBA have a lot in common
Both methods are like blenders: all things of value, including costs (value consumed) and benefits (value created) are valued in a common unit of measurement (money) and aggregated so that they can be reduced to a single indicator.
SROI and CBA might look as if they’re about money - but money is just a common unit of valuation and a convenient intermediary, which is also why we use it to buy and sell things. Both methods are fundamentally interested in the value people and groups place on a resource use and its impacts. SROI calls this “social value” while CBA calls it “social welfare” or sometimes “wellbeing”. These terms each carry slightly different baggage in terms of their underlying constructs and ways to measure them, but they’re all trying to answer the same basic question: Does this investment or intervention make society better off overall?
It’s my view that both SROI and CBA offer principles and practices that could strengthen the other. Either method can be used in ways that inform or misinform. Neither method is sufficient to provide a comprehensive evaluation of a complex social investment on its own, but a high quality CBA or SROI can contribute important information to such an evaluation.
Similar, but not identical
The Guide to SROI provides helpful, practical advice on designing and undertaking a study using this approach. The basic steps in SROI are quite similar to the steps involved in doing a CBA, as specified in guides such as the UK Green Book. Both approaches involve identifying, quantifying, attributing, monetising, discounting (to adjust for timing), and aggregating the incremental costs and benefits of a project, intervention, program, or policy. Both produce the same indicators, such as Net Present Value (NPV) and Benefit Cost Ratio (BCR).
But SROI and CBA practitioners tend to emphasise different parts of the process. CBA guidance focuses on technical aspects of the methodology, aimed at ensuring precision, accuracy, and reliability - for example, addressing issues of attribution, valuation, discounting, and robust ways of dealing with uncertainty. SROI places comparatively greater emphasis on an important social aspect of good evaluation, working with stakeholders to understand what they value.
SROI has a set of eight principles, summarised below (the 8th one being a recent addition). These are good principles to keep in mind for any evaluation.
The SROI Guide makes some claims about how it differs from CBA:
One difference between SROI and economic appraisal as described in HM Treasury’s Green Book is that SROI is designed as a practical management tool that can be used by both small and large organisations, rather than from a macro perspective. SROI focuses on, and emphasises, the need to measure value from the bottom up, including the perspective of different stakeholders, while the Green Book appraisal is about valuing costs and benefits to the whole of UK society. (SROI Network, 2012, p. 95).
In my opinion this over-states the differences, because either method can be used by anybody in any setting, at any scale from whole of society down to an individual project, and from either a top-down (macro) or bottom-up (micro) perspective. Nonetheless, it is probably true that some sectors (such as philanthropy and social impact) have been more enthusiastic in their adoption of SROI, while others (such as for-profit and government decision-makers) have tended to use CBA.
Either method can be (and is) used to look forward (ex-ante) to inform decisions, or backward (ex-post) to assess whether costs and benefits turned out as expected. In practice, CBA has been more widely used for ex-ante assessment of the potential value of proposed interventions than for ex-post evaluation of established interventions.
CBA has been around longer than SROI
SROI first emerged in the mid-1990s, from a social accounting tradition. In contrast, CBA is based on principles of welfare economics that were developed in the late 1700s and early 1800s. The origins of CBA itself trace back to the French economist Jules Dupuit (1804-1866), Chief Engineer for the City of Paris, whose seminal works were guided by the principle that “the only utility is that which people are willing to pay for”. Dupuit’s ideas were further developed by British neoclassical economists Alfred Marshall and A.C. Pigou in the early 20th century. In the Second World War CBA was used for military purposes. During the 1960s and 1970s its use spread across many areas of public policy.
CBA now has a well established and very large professional community. In any major city of the world you can find university courses and other training in CBA. There’s an extensive and ever-growing body of books and peer reviewed articles on the theory and practice of CBA.
CBA is widely used to inform financial decision-making, where records or forecasts of capital investments, operating costs, and revenues generated, and analysis of different scenarios, can support investment and operational decisions based on estimated risk and return on investment. This is sometimes called “financial CBA”.
However, CBA provides a conceptual framework that is much more than a financial analysis tool. For example, CBA is widely used in policy making in sectors as diverse as transport, infrastructure, healthcare, education, community services, and more. The scope of a CBA can be as broad as that of SROI. When a CBA is conducted with the intent of including intangible social value (i.e., social benefits and/or costs), it is sometimes called “social CBA”.
Under both SROI and CBA, there are multiple methodological options and a fair bit of flexibility for analysts to decide what is in and out of scope, how to deal with attribution, and different ways to estimate monetary values of things. Both fields of practice will continue to undergo evolution and refinement. However, possibly because CBA has had a longer duration to develop, there are currently more detailed methodologies for conducting CBA.
Balancing accessibility with rigour
SROI has introduced systematic analysis of costs and benefits to a wider audience than those who are trained in economic evaluation. This is positive, but also has risks.
As SROI is at an earlier stage of development than CBA, and may be used by a wider range of people with wider variability in training and skills, there may be a greater chance that if two people were conducting SROIs on the same program they might come to substantially different findings.
A recent article on SROI highlighted some deficiencies in the way some SROIs have been carried out. For example:
Causal attribution of benefits to the investment or intervention being evaluated was, in SROI, highly variable in its reliability and often failed to meet established standards of scientific research. (In comparison, social CBA places greater emphasis on identifying only benefits and costs that can credibly be attributed to the programme being evaluated).
SROI has sometimes taken significant latitude in valuing impacts, with varying reliability. (Social CBA takes a more conservative approach to including benefits and costs that can be translated into monetary values with a reasonable degree of certainty).
SROI is vulnerable to false precision, where the output of a precise-looking ‘return on investment’ number obscures inherently imprecise estimates that are based on a range of assumptions. (A good social CBA would include scenario analysis to produce a range of estimates, transparent assumptions, and sensitivity analysis to understand how uncertainty might affect conclusions).
Note that these are not shortcomings of SROI per se, but rather indicate variability in they way it is practiced. These are the kinds of challenges to be expected when introducing any method to a new audience.
These challenges are aptly summarised in a quote I read recently in Cameron Norman’s blog (I’m using it out of its original context, but it’s equally applicable here):
Making methodology accessible while maintaining a professional level of rigor is a delicate balancing act. Doing it well requires practitioners to become comfortable providing access to tools while also protecting against the temptation to equate process-following with methodology. (Ruth Schmidt, Three Tensions Behavioral Design Must Navigate)
In this regard, I contend that SROI could be improved if analysts follow accepted good practices developed over decades by theorists and practitioners of CBA (indeed, the best SROIs I’ve seen were done by economists).
CBA, in turn, could pay greater attention to engaging directly with stakeholders, in keeping with SROI principles and, as I argued in a recent paper, with program evaluation standards.
Potato / Potāto
Ultimately, if CBA and SROI were both being used to their full potential, I believe they would look the same and would reach the same conclusions. Differences in conclusions reached between any two studies are likely to reflect departures from good practice rather than inherent problems with the methods themselves.
Whichever approach you use, be explicit about:
Identifying and engaging stakeholders in order to understand their context and what they value
Not mandating or imposing any method of evaluation as a gold standard - and if CBA or SROI are agreed to be an appropriate method for the situation, then…
Defining the perspective of the study (value to whom) and possibly carrying out the analysis from more than one perspective (e.g., to understand how costs and benefits affect different groups)
Defining the scope of the study (which costs and benefits are in or out)
Selecting an appropriate time horizon (how long costs and benefits are expected to last or matter) or a range of time horizons for scenario analysis
Selecting an appropriate discount rate (to adjust the value of costs and benefits according to their timing) or a range of discount rates
Selecting defensible methods for estimating monetary values for intangible benefits and costs (bearing in mind that different methods can give different estimates) or a range of monetary values
Analysing impacts as the difference between outcomes with and without the intervention or investment being evaluated, and taking a robust approach to causal inference that does not start with assuming that the intervention caused or contributed to a set of outcomes
Carrying out sensitivity analysis (to understand the implications of key assumptions and how they affect the conclusions reached by the study), and scenario analysis (to understand the range of potential results under different sets of conditions, e.g., optimistic/mid-range/pessimistic)
Reporting the study design, data, assumptions, and procedures in a transparent way that makes the results replicable and challengeable
Transparently reporting limitations of the study
Conducting the study with integrity, in line with program evaluation standards and professional evaluation codes of ethics.
Quality assurance
If you’re reading a SROI or CBA report, you can use the list above to consider how well it was carried out and how credible you find it. Systematic analysis of costs and benefits can be used to provide rigorous evaluations. It can also be used for marketing and persuasion. Can you tell them apart?
Not enough on their own
Used well, either CBA or SROI can offer insights you can’t get any other way. The ability to measure and compare value created and consumed in the same units is unique and powerful.
But aggregating monetary valuations of benefits and costs is often too narrow a basis to comprehensively answer evaluative questions like:
Is this program/policy good use of resources?
Is it creating enough value?
How could more value be created from the resources invested?
If these are the sorts of questions you’re seeking to answer, I recommend incorporating insights from social CBA or SROI within a broader evaluation framework that includes evaluative reasoning, mixed methods and stakeholder participation.
If you enjoy reading these posts please drop me a line! I welcome comments, questions, or suggestions for future topics.
Thanks to Aaron Schiff for peer review. Any errors or omissions are my own.
Thanks Julian for another insightful piece! I am an avid reader of your posts! Take care