Developing Guidance on Forest Governance Data Collection for Assessment and Monitoring

CHALLENGE
Good forest governance has a central role in achieving sustainable forest management. It is also critical to ensure the effectiveness of plans to reduce greenhouse gas emissions from deforestation and forest degradation in developing countries (REDD), as well as to ensure the effectiveness of efforts to reduce illegal activities in the forest sector.
 

Assessment and monitoring of governance are essential tools in promoting reforms to achieve better forest governance. Making available a compendium of evidence-based approaches to forest governance data collection could help countries decide between different options as they respond to various requirements, for example, such as those arising from the Voluntary Partnership Agreement (a bilateral trade agreement between the EU and some timber-producing countries) or from REDD+.
 

APPROACH
PROFOR has developed a common framework for assessing and monitoring forest governance, in collaboration with the Food and Agriculture Organization of the United Nations and other partners, including the World Resources Institute, Chatham House, the European Forest Institute, and the UN-REDD Programme. PROFOR also piloted an approach for collecting data, based on questionnaires scored by multiple stakeholders, which so far has been used in a range of countries: Uganda, Burkina Faso, Kenya, Russian Federation, Madagascar, Democratic Republic of Congo and Liberia. However, this approach is only one of many indicator and data collection options available to practitioners.

To improve the process, PROFOR has a new project that aims to compile a compendium of approaches to forest governance data collection, to provide practitioners with a menu of options that is customizable to specific contexts. The guide on data collection approaches will be accompanied by a library of forest data collection initiatives that have been field-tested in one or more countries. To the extent possible, the guide and library will be updated as additional field experiences become available.

The production of this menu of options is expected to strengthen collaboration and cooperation between the World Bank and its partners in the area of forest governance diagnostics, to foster agreement on common indicators for forest governance, and to reduce multiple reporting.

RESULTS
In June 2012, in Rome, thirty-five international and national experts gathered to discuss common issues in governance assessment and monitoring, and considered the value of producing resource materials for people measuring forest governance. The meeting resulted in the need for developing guidance for forest governance data collection and the creation of the Core Expert Group on Forest Data Collection. In November 2012, the first core group meeting took place in Brussels and produced an outline of the guidance—“a practical guide to measuring forest governance for assessments and monitoring”—and a plan for collecting tools and cases in this area. In June 2013, PROFOR, FAO, the UN-REDD Programme, WRI and others convened a second core group meeting. This meeting reviewed a draft guide based on the outline and discussed next steps.

The guide was launched in June 2014, at COFO 22, in Rome. This guide presents a step-by-step approach to planning forest governance assessment or monitoring, collecting data, analyzing it, and making the results available to decision makers and other stakeholders. It also presents five case studies to illustrate how assessment or monitoring initiatives have applied the steps in practice, and it includes references and links to dozens of sources of further information.

For stories and updates on related activities, follow us on Twitter and Facebook, or subscribe to our mailing list for regular updates.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : Phil Cowling, Kristin DeValue, and Kenneth Rosenbaum
Last Updated : 06-16-2024

A Crystal Ball for Forests: Using Today's Indicators to Predict Tomorrow's Impacts

Worldwide, an estimated 1.3 billion people - the majority of whom subsist on less than $1.25 per day - depend directly on forests for their livelihoods.[i] Therefore, it would seem clear that programs to restore and maintain forests must also contribute to poverty alleviation. But is this true, and if so, how do we know?

On one hand, there is increasing demand for better evidence on program effectiveness, including in the forestry sector. On the other hand, many programs – particularly those focused on natural resource-based development – can only be assessed properly by looking at long-term outcomes. As a result, there is a mismatch between the urgency with which global poverty and deforestation issues need to be addressed, and the time that it takes to get results.

To help close this knowledge gap, a World Bank team received PROFOR funding to develop a set of Predictive Proxy Indicators (PPIs) – measures of progress made while a project or policy is still ongoing, but that provide a forecast for a longer-term impact. PPIs are often used in other fields: for example, education level is widely used as a predictor for future earnings. But use of PPIs is a new development for the forestry sector, with the potential not just to measure the impacts of forestry programs on poverty reduction and economic growth, but also on other important development outcomes, such as biodiversity conservation, climate change mitigation and adaptation, and good governance.

“This is a long-standing issue with evaluating forestry projects,” notes lead author and Senior Forestry Specialist Daniel Miller. “It takes years for trees to mature to a point where they deliver environmental benefits, let alone economic ones, but funding cycles are much shorter so program evaluation may not reflect the benefits that accrue later on. Predictive proxies are a cost-effective complement to rigorous impact evaluation, with the potential to address the issue of different time lags.”

To maximize the accuracy of their predictive power, each PPI is composed of a cluster of indicators that, taken together, was determined to have strong predictive potential. In addition, each PPI is based on a plausible theory of change that explains why the PPI was likely to predict a certain outcome as a result of an intervention. For instance, the PPI for Sustainable, Forest-Related Income, which has poverty reduction as an objective, is made up of three indicators, as seen in the following figure:

One of the report’s main findings was that PPIs for forestry do in fact exist and can be used in practice. This, according to Miller, was not at all a foregone conclusion at the start of the project: “Reliable outcome indicators are usually simple to use once they’ve been developed, but identifying them can be difficult. We began this research knowing that we might fail to find anything functional. We’d never seen the PPI approach used in forestry before, so I think it’s an innovative way of solving a fundamental challenge in the sector.”

Encouragingly, the Core Sector Indicators (CSIs) already used by the World Bank have strong potential as PPIs, meaning that CSIs can help to capture not only end-of project outcomes, but also longer-term impacts of forest investments, and in a consistent way across countries and contexts. According to the report, another reason to use detailed monitoring and evaluation (M&E) strategies such as PPIs is that M&E investments are associated with higher project success: Projects with a highly satisfactory M&E component were more likely to have a highly satisfactory outcome rating too, compared to a project with only a satisfactory M&E score.  

PPIs are already being integrated into new World Bank projects, such as the Argentina Forests and Community Project. This initiative aims to increase access to markets and basic services by small forest producers, including indigenous people and campesinos, in Argentina’s comparatively poor but forest-rich Northern provinces. Activities will focus on helping communities to develop Sustainable Forest Management (SFM) plans and to strengthen their tenure. To assess effectiveness, the project team will collect data on the three indicators under the Sustainable Forest-Related Income PPI. The project will also be assessed using rigorous impact evaluation methods, making it the first forward-looking test case of how well the forestry PPIs perform as predictors of future outcomes.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Last Updated : 06-16-2024

Participatory Benefit Sharing for Forest-Dependent Communities

CHALLENGE
Good forest governance has a central role in achieving sustainable forest management. It is also critical to ensure the effectiveness of plans to reduce greenhouse gas emissions from deforestation and forest degradation in developing countries (REDD), as well as to ensure the effectiveness of efforts to reduce illegal activities in the forest sector.

Assessment and monitoring of governance are essential tools in promoting reforms to achieve better forest governance. Since early efforts starting in the 1990s, quality assessments of forest governance have gradually relied less on international experts and more on national institutions and local expertise. Purely technical approaches are giving way to better integration of political and managerial issues. Meanwhile, quality assessments of forest governance are assuming greater significance in the context of discussions of global climate change.

A compendium of evidence-based approaches to forest governance data collection could help countries decide between different options as they respond to various requirements, for example arising from the Voluntary Partnership Agreement, a bilateral trade agreement between the EU and some timber-producing countries.

APPROACH
PROFOR has developed a common framework for assessing and monitoring forest governance, in collaboration with the Food and Agriculture Organization of the United Nations and other partners, including the World Resources Institute, Chatham House, the European Forest Institute, and the UN-REDD Programme. PROFOR also piloted an approach for collecting data, based on questionnaires answered by multiple stakeholders, which so far has been used in a range of countries: Uganda, Burkina Faso, Kenya, Russian Federation, Madagascar and Democratic Republic of Congo. However, this approach is only one of many indicator and data collection options available to practitioners.

To improve the process, PROFOR has a new project that aims to produce a compendium of approaches to forest governance data collection, to provide practitioners with a menu of options that is customizable to specific contexts. The guide on data collection approaches will be accompanied by a library of forest data collection initiatives that have been field-tested in one or more countries. To the extent possible, the guide and library will be updated as additional field experiences become available.

The production of this menu of options is expected to strengthen collaboration and cooperation between the World Bank and its partners in the area of forest governance diagnostics, to foster agreement on common indicators for forest governance, and to reduce multiple reporting.

RESULTS
In June 2012 in Rome, thirty-five international and national experts gathered to discuss common issues in governance assessment and monitoring, and considered the value of producing resource materials for people measuring forest governance. The meeting resulted in the need for developing guidance for forest governance data collection and the creation of the Core Expert Group on Forest Data Collection. In November 2012, the first core group meeting took place in Brussels and produced an outline of the guidance—“a practical guide to measuring forest governance for assessments and monitoring”—and a plan for collecting tools and cases in the area.

On 20 and 21 June 2013, PROFOR, FAO, the UN-REDD Programme, WRI and others convened a second core group meeting. This meeting reviewed a draft guide based on the outline and discussed next steps.

This activity is ongoing. Findings will be shared on this page when they become available. Follow us on twitter (www.twitter.com/forestideas) or subscribe to our mailing list for regular updates.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : The World Bank,PROFOR,FAO,World Resourse Institute
Last Updated : 06-16-2024

Evaluating the Effectiveness and Impact of Forest Certification

CHALLENGE
Sustainable forest management certification enjoys wide support; yet critical evaluation of forest certification has not been carried out. Most studies that have attempted to study the impact of SFM certification have methodological shortcomings, such as selection bias (non-random participation), or lack a robust understanding of market impacts of certification at firm and farm levels. With these deficiencies, its not possible to assess the impacts of  forest certification on forests, firms, employees, and rural communities with any scientific precision.

APPROCAH
This activity financed a three-day workshop on evaluating the impacts of certification in Montpellier, France, on 12-14 November 2011.  Participants representing the evaluation community, economists, sociologists, ecologists, and foresters discussed and decided on an integrated evaluation method to elucidate whether certification is achieving its set objectives in different countries.The workshop assembled an overview of available quantitative and qualitative approaches to impact evaluation for tropical timber certification; and provided an opportunity to assemble the expertise needed to develop a research agenda for evaluating the impact of forest certification.

FINDINGS
The working paper available on this page summarizes the state of knowledge on the conservation impact of forest certification. Key conclusions and recommendations include:

• It is essential to use systematic analyses when making decisions on forest management and practices. These include better understanding of the complex dynamics and theory of change of forest sector interventions as well as on the ultimate objectives of forest management.
• Despite not having full understanding of all the dimensions and impacts of forest management certification, it can be assumed that certified forests are, on average, most likely better managed than noncertified forests. Certification also ensures to the public that sustainability of forest management is promoted.
• Forests provide several benefits. Forest management certification mainly deals with the productive functions of the forests and with ensuring that these functions can create economic goods without jeopardizing social benefits and environmental sustainability. National governments and donor agencies should continue their support for well-managed sustainable production forestry, and forest certification is one instrument for achieving it.
• Methodological challenges in measuring the impact of forest certification cannot be interpreted as lack of impact. Forest certification increases the information available in the marketplace for all participants, and this improved information increases confidence in certified producers. Having better and scientifically verified information on the impact would improve the information base even further and could also be used to improve the design of certification schemes.

RESULTS
The workshop and working paper available on this page have contributed to defining a research project on the field impacts of SFM certification conducted by CIFOR, the University of Florida and CIRAD. Implementation of the research will require extensive support as well as collaboration and sharing of experiences by a wide range of stakeholders and institutions.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : Center for International Forestry Research (CIFOR), Claudia Romero (University of Florida, USA), Centre de Coopération Internationale en Recherche Agronomique pour le Développement (CIRAD/France).
Last Updated : 06-15-2024

Developing Proxy Indicators to Assess Forests Sector Impacts

CHALLENGE

The international development community is increasingly demanding better evidence regarding the effectiveness of policies and programs across different sectors, and the forest sector is no exception. Governments and donor agencies explicitly seek to link investment to proven impact. Yet, the evidence base necessary to inform policies and programs in the sector that can successfully enhance the livelihoods of forest-dependent poor populations, foster economic growth, reduce emissions from deforestation and degradation, and conserve forest biodiversity remains weak. There is a particular need to identify robust, yet practical, indicators to track and assess the impacts of forest-related investments.

Any attempt to identify such indicators must grapple with two particular challenges in the forest sector. First, forest-related interventions are usually complex, with forestry policies, programs and projects often including multiple objectives, requiring the integration of socioeconomic and ecological expertise, and entailing processes that unfold over different spatial scales. Second, such interventions often take a long time to show results. For example, the results of investments in thinning, tree stand improvement or natural regeneration under sustainable forest management are unlikely to be evident for 10 to 30 years. These characteristics make attribution of impacts to specific interventions (as opposed to other potential factors) especially difficult within the forest sector.

APPROACH

This activity will build from the extensive evidence accumulated within the World Bank’s forests portfolio, PROFOR activities and other relevant donor-supported initiatives to address these challenges and develop guidance on the identification and use of proxy indicators for project impacts. In so doing, the Activity seeks to increase understanding among policymakers, practitioners and applied researchers of the potential short-term proxy indicators for longer-term impacts of forest sector investments and how they may be used in practice.

The Activity began by undertaking an inventory of indicators used by key actors in the sector and reviewing the available literature on proxy indicators (sometimes referred to as “lead” or “predictive” indicators). An in-depth review of the World Bank portfolio of forest operations under the current Forests Strategy (2002-present) and a relevant subset of the PROFOR portfolio lies at the heart of this Activity. Results will provide the core evidence base for a report on predictive proxy indicators (PPIs) and the conditions in which they are likely to be valid. Through this review, the activity will also analyze potential constraints to the use of indicators in donor-supported forest interventions and suggest ways these constraints might be addressed.

RESULTS

The Working Paper "Understanding Long-Term Impacts in the Forest Sector: Predictive Proxy Indicators" is now available (download at left). The main overall finding is that predictive proxy indicators do appear to exist and can be used in practice. Given the complexity and diversity of the forest sector and novelty of the task, this conclusion was far from a certainty when this research began. The authors identified a range of potential PPIs, several of which have already been used in World Bank forest projects. These PPIs focused primarily at the Project Development Objective (PDO) and Global Environment Objective level, although some intermediate-level indicators were identified.

The report argues that robust PPIs can provide an important ex-ante evaluation tool that enables practitioners and researchers to predict future outcomes and longer-term impacts if certain assumptions hold. To help inform the design and implementation of forestry operations and knowledge, we developed a list of top ranking indicators based on an assessment of their predictive potential and their SMART score. These indicators are presented in an indicator menu organized by major objective (that is, relating to poverty, biodiversity, climate, or governance) and including brief notes on how they might be used. The indicator menu is presented as an annex and represents a key product stemming from this analytical work.

This review did not identify any standalone “silver bullet” predictive proxies, but a major contribution is the idea that multiple indicators, considered together, can have strong predictive potential. The report describes a series of seven indicator clusters that form PPIs. 

PPIs are already being integrated into new World Bank projects, such as the Argentina Forests and Community Project. This initiative aims to increase access to markets and basic services by small forest producers, including indigenous people and campesinos, in Argentina’s comparatively poor but forest-rich Northern provinces. Activities will focus on helping communities to develop Sustainable Forest Management (SFM) plans and to strengthen their tenure. To assess effectiveness, the project team will collect data on the three indicators under the Sustainable Forest-Related Income PPI. The project will also be assessed using rigorous impact evaluation methods, making it the first forward-looking test case of how well the forestry PPIs perform as predictors of future outcomes.​

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : The World Bank, PROFOR
Last Updated : 06-15-2024

Assessing Impacts of Forest Governance Interventions

CHALLENGE

The World Bank’s 2002 Forest Strategy laid down two bold targets, which it saw as the collective outcome of global efforts to promote Sustainable Forest Management, through improving forest governance. First, by 2012/13, a reduction of global illegal logging by 50 percent (from an estimated baseline value of $10 billion per annum); and, second, a 50 percent decrease in the estimated value of taxes, fees, and levies willfully evaded. However, there has been little systematic effort to assess the extent to which these targets have been achieved. This is a drawback (applicable not only to the World Bank but to other development agencies as well) as it limits our ability to learn from evidence and to apply the learning in designing effective interventions.

APPROACH

By looking at a suite of the Bank’s forestry programs and projects (which have significant forest governance components), this report identifies the bottlenecks to improve tracking the impacts of forest governance interventions and suggests ways in which they can be removed and the capacity for impact evaluation (IE) strengthened.

FINDINGS AND RESULTS

The report finds that most Bank-financed projects track impacts in forest governance projects almost exclusively through log-frame approaches such as a Results Framework or a Policy Matrix. These have been used to measure progress toward project objectives through the use of performance indicators, coupled with baseline surveys and proposed target values. However, the ex-ante approaches to monitoring and evaluation in Bank projects typically do not try to establish attribution, nor do they systematically track spill-over effects (positive or negative) and leakages resulting from project interventions. They also do not consider the role of “confounding factors” that is, non-project influences, which can influence expected project outcomes. Finally, in most cases impacts are not monitored beyond the life-cycle of the project. Because of these shortcomings, the Policy Matrix or Results Framework approaches do not fully measure the impacts of Bank interventions.

The current gaps in being able to measure impacts systematically should not be taken to mean that Bank-financed interventions have not had any impacts. However, the need is to develop a culture of measurement whereby impacts are objectively and routinely measured and the learning potential through IE maximized. To this end the report recommends three broad actions:

• Develop a compendium of practical techniques for IE in forestry and raise awareness among project task team leaders. This would demonstrate the value of starting with causal pathways linking the intended outcomes to the necessary inputs and outputs;
• Enhance resources and improve opportunities to embed IE in project design and implementation. The Bank should consider “up-streaming” a discussion of tracking project impacts at the Project Concept Note stage; and,
• Establish a community of practice (CoP) and a help-desk on IE for forestry. Establishing a CoP within and outside the Bank can increase member knowledge by sharing information and experiences to allow for an effective exchange of learning.

Because poverty reduction, improvements in the security of livelihoods, conservation of wildlife and biodiversity, and cross-sectoral collaboration, to name some objectives, go hand-in-hand with interventions to improve forest governance, this report recommends that evaluation approaches should track all activity impacts. Thus, the three actions suggested above should consider tracking impacts more widely than for forest governance alone.

The report acknowledges the main limitations of the analysis. The experiences and the data are all from one institution—the World Bank—and (including as it does, twenty programs and projects) are limited in coverage. Thus, caution has to be exercised in any attempt to draw out general lessons. Nevertheless, this report provides a useful first cut contribution to the challenge of assessing the impacts of forest governance interventions and of assessing impacts more generally. Future work should emphasize collaborative exploration (among development partners assisting with sustainable forest management and key client countries) as a way to build up the evidence base on cost-effective and easy to replicate impact evaluation techniques and to rapidly build up a compendium of practical approaches.

For stories and updates on related activities, follow us on twitter and facebook , or subscribe to our mailing list for regular updates.

Author : World Bank
Last Updated : 06-15-2024