SROI to IMN Case Study 1: Local Value Chain Development In Indonesia

The value of changes brought about by intervention activities has attracted many, and Social Return on Investment (SROI) and Impact Management Norms (IMN) are two commonly used standards for value measurement. This project is designed to convert reports applying SROI into ones following IMN, believing that experience and knowledge learnt will improve measurement methods shortly.

Flotim (Flores Timur) is a regency in East Nusa Tenggara province of Indonesia and is also known for its natural environment favorable for horticulture, agriculture, marine industry, and fisheries. However, Residents are predominantly smallholder farmers and still live in poverty. Studies revealed that the lack of access to markets and market information were the two most important factors inhibiting growth for smallholder farmers.

Project description

To address the issue above, World Vision Indonesia carried out the Local Value Chain Development (LVCD) project from April 2009 to March 2012 in 16 villages on Flores Island. The project received US$116,000 in direct funding over three years, and the cost of the technical, design, monitoring, and evaluation support was US$34,000 over three years. Strategies employed in the LVCD project included (1) mobilizing farmers into producer groups; (2) training producer groups in core business and marketing skills;(3) facilitating the involvement of producer groups in value chain research; and (4) coaching on negotiation skills, accessing market information and opportunities, understanding prices, relationship building and networking with service providers.

Analysis

The original assessment report applying the Social Return on Investment (SROI) analysis approach is turned into a five-dimension table below, in alignment with the framework of Impact Management Norms (IMN).

In the analysis report completed in 2014, four types of stakeholders were involved in the LVCD project, and five positive results were derived from it. The two changes experienced by local farmers, Improved Social and Economic Well-being and Sense of Empowerment, have the highest value. In this conversion report, to limit the scope of discussion, the focus is on local farmers and the two changes they experienced. 

The WHAT dimension, the first dimension of the IMN framework, explains the outcomes resulting from intervention activities and involves five categories of data: outcome level in period, outcome indicator, outcome threshold, importance of outcome to stakeholders, and SDG. These data correspond to the second step in analyzing using the SROI approach, which is mapping outcomes. Chapter 8 of the original report describes the changes experienced by stakeholders. However, except for the outcome indicator, this chapter does not provide further information on the outcome level in period, outcome threshold, and importance of outcome to stakeholders. Therefore, these fields in Table 1 below are all marked as “n/a.” A key intervention in the LVCD project was the employment of a Market Facilitator who empowered local farmers with strategies responsive to market demands, resulting in revitalized local producer groups and improved negotiation abilities for better product prices. This intervention yielded significant outcomes for local farmers, notably in improving social and economic well-being and a sense of empowerment. While certain elements like the outcome threshold and importance to stakeholders are not specified, it’s clear that the Market Facilitator’s intervention significantly contributes to SDG 1: No Poverty.

Well-defined Outcome		Improved social and economic Well-being	Sense of empowerment	Increased leadership and project management knowledge and skills	Increased profitability	Increased understanding of program effectiveness
WHAT	Outcome Level in Period	n/a
	Outcome Indicator	The amount of money spent on the various social and economic dimension of their lives	The number of farmers who self-reported that they felt more empowered as a result of the LVCD project	The number of local market facilitators who participated in LVCD training workshops	The reduction in the costs involved in buying farm products	The amount of money reallocated from one ADP project, which was not achieving its stated goal, to the LVCD project, which the ADP staff perceived to be showing results and achieving its goal.
	Outcome threshold	n/a
	Importance of Outcome to Stakeholders	n/a
	SDG	SDG 1 No Poverty

The WHO dimension, the second dimension of the IMN framework, describes those who are affected by intervention activities and includes four data categories: stakeholder, geographic boundary, outcome level at baseline, and stakeholder characteristics. In terms of content, these data correspond to the first step in analyzing using the SROI approach, which is establishing scope and identifying stakeholders. Chapter 4 of the original report may provide these data, which lists the stakeholders involved in the discussion and explains why some stakeholders were excluded. As shown in Table 2, the stakeholders impacted by the intervention activities include, local farmers, market facilitators, buyers, and Flotim Area Development Program (ADP) staff. Specifically, the interventions engaged 1500 local farmers and 40 market facilitators. However, Due to the lack of information on stakeholder characteristics, these stakeholders cannot be further subdivided into multiple subgroups. Additionally, data pertaining to the outcome level at baseline is not provided in this chapter, hence the “n/a” notation in Table 2 for these fields.

Well-defined Outcome		Improved social and economic Well-being	Sense of empowerment	Increased leadership and project management knowledge and skills	Increased profitability	Increased understanding of program effectiveness
WHO	Stakeholders	Local Farmers (1500 persons)		Market Facilitators (40 persons)	Buyers	Flotim Area Development Program (ADP) Staff
	Geographical Boundary	16 (out of 36) Districts, Flotim (Flores Timur) Regency, Nusa Tenggara Timur (NTT) Province, Indonesia.
	Outcome Level at Baseline	n/a
	Stakeholder Characteristics	n/a

The HOW MUCH dimension, the third dimension of the IMN framework, describes the importance of outcomes derived from intervention activities, involving three data categories: scale, depth, and duration. In terms of content, these data correspond to the third step in analyzing using the SROI approach, which is Evidencing outcomes and giving them a value. Chapter 9 of the original report may provide these data, which clarified the number of stakeholders (scale) and the degree of changes they experienced (depth). However, due to incomplete data, certain columns in Table 3 are labeled as “n/a..” In Table 3, the “scale” reflects the reach of each outcome. Improved social and economic well-being impacted an average of 910 individuals, while a sense of empowerment reached an average of 683 individuals. The “depth” demonstrates the degree of change experienced by the beneficiaries, noted as an average annual income increase of 44.7 USD for those experiencing improved social and economic well-being. The “duration” data, or the anticipated length of the intervention’s impact, is detailed in Chapter 10 of the original report. In that chapter, it is estimated that the effects of the intervention activities can last for 5 to 10 years, which is suspected of being exaggerated compared to the current practice of SROI analysis.

Well-defined Outcome		Improved social and economic Well-being	Sense of empowerment	Increased leadership and project management knowledge and skills	Increased profitability	Increased understanding of program effectiveness
HOW MUCH	Scale	910 persons on average (2732 persons in total)	683 person on average (2050 persons in total)	22 persons on average (67 persons in total)	4 persons on average (12 persons in total)	n/a
	Depth	Annual income increase by 44.7 USD on average	n/a	n/a	n/a	n/a
	Duration	5 to 10 years

The CONTRIBUTION dimension, the fourth dimension of the IMN framework, clarifies whether and to which extent intervention activities lead to outcomes, involving four data categories: deadweight, attribution, displacement, and dropout. In terms of content, these data correspond to the fourth step in analyzing using the SROI approach, which is Establishing impact. Chapter 10 of the original report may provide these data. As shown in Table 4 below, deadweight, displacement, and attribution are estimated at zero percent, asserting that the project’s outcomes are fully due to its implementation, with no displacement of changes occurring as a result. Drop-off refers to the longevity of project outcomes post-intervention, which stakeholders projected to continue for over 10 years. However, a conservative 25% drop-off rate was applied to account for the future influence of other factors. This assessment is suspected of being exaggerated. After all, several coexistent factors usually lead to the same change in people’s lives.  

The RISK dimension, the final dimension of the IMN framework, explains possible risks of intervention activities, involving two data categories: type and level. In the IMN framework, type is categorized into nine categories, while the level is classified into three types: high, medium, and low. None of the six stages of SROI directly address differentiating risk types and levels. Due to the analyst’s mention of several potential flaws in the original report (see page 11), these flaws, in terms of content, can relate to the Stakeholder Participation Risk. Considering that this risk can potentially impact the assessment results, the corresponding cell in Table 4 is labeled as ‘High.’

Well-defined Outcome		Improved social and economic Well-being	Sense of empowerment	Increased leadership and project management knowledge and skills	Increased profitability	Increased understanding of program effectiveness
CONTRIBUTION	Deadweight	0%
	Attribution	0%
	Displacement	0%
	Dropout	0%
Risk	Type	Stakeholder Participation Risk
	Level	High

This analysis report, completed in 2014, adopted the emerging Social Return on Investment (SROI) analysis approach to present the social value derived from the LVCD project. After several years of reform and refinement, the SROI analysis approach has gradually improved, and as a result, the number of information required to be presented increases. However, overall, this analysis report does not meet the requirements set in accordance with the current SROI approach. Among the shortcomings of the analysis report, the four factors related to the judgment of the degree of project contribution may easily attract criticism. At the same time, viewing from the IMN structure, important information such as outcome level in period, outcome threshold, and some others are absent in the main text, which affects readers’ judgment on the completeness of this analysis. In summary, this is an early report applying the SROI analysis approach to present the impact of a project implemented in Indonesia. There is merit in this regard, but there is still much room for improvement.