CHALLENGE

The World Bank’s 2002 Forest Strategy laid down two bold targets, which it saw as the collective outcome of global efforts to promote Sustainable Forest Management, through improving forest governance. First, by 2012/13, a reduction of global illegal logging by 50 percent (from an estimated baseline value of $10 billion per annum); and, second, a 50 percent decrease in the estimated value of taxes, fees, and levies willfully evaded. However, there has been little systematic effort to assess the extent to which these targets have been achieved. This is a drawback (applicable not only to the World Bank but to other development agencies as well) as it limits our ability to learn from evidence and to apply the learning in designing effective interventions.

APPROACH

By looking at a suite of the Bank’s forestry programs and projects (which have significant forest governance components), this report identifies the bottlenecks to improve tracking the impacts of forest governance interventions and suggests ways in which they can be removed and the capacity for impact evaluation (IE) strengthened.

FINDINGS AND RESULTS

The report finds that most Bank-financed projects track impacts in forest governance projects almost exclusively through log-frame approaches such as a Results Framework or a Policy Matrix. These have been used to measure progress toward project objectives through the use of performance indicators, coupled with baseline surveys and proposed target values. However, the ex-ante approaches to monitoring and evaluation in Bank projects typically do not try to establish attribution, nor do they systematically track spill-over effects (positive or negative) and leakages resulting from project interventions. They also do not consider the role of “confounding factors” that is, non-project influences, which can influence expected project outcomes. Finally, in most cases impacts are not monitored beyond the life-cycle of the project. Because of these shortcomings, the Policy Matrix or Results Framework approaches do not fully measure the impacts of Bank interventions.

The current gaps in being able to measure impacts systematically should not be taken to mean that Bank-financed interventions have not had any impacts. However, the need is to develop a culture of measurement whereby impacts are objectively and routinely measured and the learning potential through IE maximized. To this end the report recommends three broad actions:

• Develop a compendium of practical techniques for IE in forestry and raise awareness among project task team leaders. This would demonstrate the value of starting with causal pathways linking the intended outcomes to the necessary inputs and outputs;
• Enhance resources and improve opportunities to embed IE in project design and implementation. The Bank should consider “up-streaming” a discussion of tracking project impacts at the Project Concept Note stage; and,
• Establish a community of practice (CoP) and a help-desk on IE for forestry. Establishing a CoP within and outside the Bank can increase member knowledge by sharing information and experiences to allow for an effective exchange of learning.

Because poverty reduction, improvements in the security of livelihoods, conservation of wildlife and biodiversity, and cross-sectoral collaboration, to name some objectives, go hand-in-hand with interventions to improve forest governance, this report recommends that evaluation approaches should track all activity impacts. Thus, the three actions suggested above should consider tracking impacts more widely than for forest governance alone.

The report acknowledges the main limitations of the analysis. The experiences and the data are all from one institution—the World Bank—and (including as it does, twenty programs and projects) are limited in coverage. Thus, caution has to be exercised in any attempt to draw out general lessons. Nevertheless, this report provides a useful first cut contribution to the challenge of assessing the impacts of forest governance interventions and of assessing impacts more generally. Future work should emphasize collaborative exploration (among development partners assisting with sustainable forest management and key client countries) as a way to build up the evidence base on cost-effective and easy to replicate impact evaluation techniques and to rapidly build up a compendium of practical approaches.