A Crystal Ball for Forests: Using Today's Indicators to Predict Tomorrow's Impacts

Worldwide, an estimated 1.3 billion people - the majority of whom subsist on less than $1.25 per day - depend directly on forests for their livelihoods.[i] Therefore, it would seem clear that programs to restore and maintain forests must also contribute to poverty alleviation. But is this true, and if so, how do we know?

On one hand, there is increasing demand for better evidence on program effectiveness, including in the forestry sector. On the other hand, many programs – particularly those focused on natural resource-based development – can only be assessed properly by looking at long-term outcomes. As a result, there is a mismatch between the urgency with which global poverty and deforestation issues need to be addressed, and the time that it takes to get results.

To help close this knowledge gap, a World Bank team received PROFOR funding to develop a set of Predictive Proxy Indicators (PPIs) – measures of progress made while a project or policy is still ongoing, but that provide a forecast for a longer-term impact. PPIs are often used in other fields: for example, education level is widely used as a predictor for future earnings. But use of PPIs is a new development for the forestry sector, with the potential not just to measure the impacts of forestry programs on poverty reduction and economic growth, but also on other important development outcomes, such as biodiversity conservation, climate change mitigation and adaptation, and good governance.

“This is a long-standing issue with evaluating forestry projects,” notes lead author and Senior Forestry Specialist Daniel Miller. “It takes years for trees to mature to a point where they deliver environmental benefits, let alone economic ones, but funding cycles are much shorter so program evaluation may not reflect the benefits that accrue later on. Predictive proxies are a cost-effective complement to rigorous impact evaluation, with the potential to address the issue of different time lags.”

To maximize the accuracy of their predictive power, each PPI is composed of a cluster of indicators that, taken together, was determined to have strong predictive potential. In addition, each PPI is based on a plausible theory of change that explains why the PPI was likely to predict a certain outcome as a result of an intervention. For instance, the PPI for Sustainable, Forest-Related Income, which has poverty reduction as an objective, is made up of three indicators, as seen in the following figure:

One of the report’s main findings was that PPIs for forestry do in fact exist and can be used in practice. This, according to Miller, was not at all a foregone conclusion at the start of the project: “Reliable outcome indicators are usually simple to use once they’ve been developed, but identifying them can be difficult. We began this research knowing that we might fail to find anything functional. We’d never seen the PPI approach used in forestry before, so I think it’s an innovative way of solving a fundamental challenge in the sector.”

Encouragingly, the Core Sector Indicators (CSIs) already used by the World Bank have strong potential as PPIs, meaning that CSIs can help to capture not only end-of project outcomes, but also longer-term impacts of forest investments, and in a consistent way across countries and contexts. According to the report, another reason to use detailed monitoring and evaluation (M&E) strategies such as PPIs is that M&E investments are associated with higher project success: Projects with a highly satisfactory M&E component were more likely to have a highly satisfactory outcome rating too, compared to a project with only a satisfactory M&E score.  

PPIs are already being integrated into new World Bank projects, such as the Argentina Forests and Community Project. This initiative aims to increase access to markets and basic services by small forest producers, including indigenous people and campesinos, in Argentina’s comparatively poor but forest-rich Northern provinces. Activities will focus on helping communities to develop Sustainable Forest Management (SFM) plans and to strengthen their tenure. To assess effectiveness, the project team will collect data on the three indicators under the Sustainable Forest-Related Income PPI. The project will also be assessed using rigorous impact evaluation methods, making it the first forward-looking test case of how well the forestry PPIs perform as predictors of future outcomes.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Last Updated : 06-16-2024

Developing Proxy Indicators to Assess Forests Sector Impacts

CHALLENGE

The international development community is increasingly demanding better evidence regarding the effectiveness of policies and programs across different sectors, and the forest sector is no exception. Governments and donor agencies explicitly seek to link investment to proven impact. Yet, the evidence base necessary to inform policies and programs in the sector that can successfully enhance the livelihoods of forest-dependent poor populations, foster economic growth, reduce emissions from deforestation and degradation, and conserve forest biodiversity remains weak. There is a particular need to identify robust, yet practical, indicators to track and assess the impacts of forest-related investments.

Any attempt to identify such indicators must grapple with two particular challenges in the forest sector. First, forest-related interventions are usually complex, with forestry policies, programs and projects often including multiple objectives, requiring the integration of socioeconomic and ecological expertise, and entailing processes that unfold over different spatial scales. Second, such interventions often take a long time to show results. For example, the results of investments in thinning, tree stand improvement or natural regeneration under sustainable forest management are unlikely to be evident for 10 to 30 years. These characteristics make attribution of impacts to specific interventions (as opposed to other potential factors) especially difficult within the forest sector.

APPROACH

This activity will build from the extensive evidence accumulated within the World Bank’s forests portfolio, PROFOR activities and other relevant donor-supported initiatives to address these challenges and develop guidance on the identification and use of proxy indicators for project impacts. In so doing, the Activity seeks to increase understanding among policymakers, practitioners and applied researchers of the potential short-term proxy indicators for longer-term impacts of forest sector investments and how they may be used in practice.

The Activity began by undertaking an inventory of indicators used by key actors in the sector and reviewing the available literature on proxy indicators (sometimes referred to as “lead” or “predictive” indicators). An in-depth review of the World Bank portfolio of forest operations under the current Forests Strategy (2002-present) and a relevant subset of the PROFOR portfolio lies at the heart of this Activity. Results will provide the core evidence base for a report on predictive proxy indicators (PPIs) and the conditions in which they are likely to be valid. Through this review, the activity will also analyze potential constraints to the use of indicators in donor-supported forest interventions and suggest ways these constraints might be addressed.

RESULTS

The Working Paper "Understanding Long-Term Impacts in the Forest Sector: Predictive Proxy Indicators" is now available (download at left). The main overall finding is that predictive proxy indicators do appear to exist and can be used in practice. Given the complexity and diversity of the forest sector and novelty of the task, this conclusion was far from a certainty when this research began. The authors identified a range of potential PPIs, several of which have already been used in World Bank forest projects. These PPIs focused primarily at the Project Development Objective (PDO) and Global Environment Objective level, although some intermediate-level indicators were identified.

The report argues that robust PPIs can provide an important ex-ante evaluation tool that enables practitioners and researchers to predict future outcomes and longer-term impacts if certain assumptions hold. To help inform the design and implementation of forestry operations and knowledge, we developed a list of top ranking indicators based on an assessment of their predictive potential and their SMART score. These indicators are presented in an indicator menu organized by major objective (that is, relating to poverty, biodiversity, climate, or governance) and including brief notes on how they might be used. The indicator menu is presented as an annex and represents a key product stemming from this analytical work.

This review did not identify any standalone “silver bullet” predictive proxies, but a major contribution is the idea that multiple indicators, considered together, can have strong predictive potential. The report describes a series of seven indicator clusters that form PPIs. 

PPIs are already being integrated into new World Bank projects, such as the Argentina Forests and Community Project. This initiative aims to increase access to markets and basic services by small forest producers, including indigenous people and campesinos, in Argentina’s comparatively poor but forest-rich Northern provinces. Activities will focus on helping communities to develop Sustainable Forest Management (SFM) plans and to strengthen their tenure. To assess effectiveness, the project team will collect data on the three indicators under the Sustainable Forest-Related Income PPI. The project will also be assessed using rigorous impact evaluation methods, making it the first forward-looking test case of how well the forestry PPIs perform as predictors of future outcomes.​

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : The World Bank, PROFOR
Last Updated : 06-15-2024