When development workers congregate to discuss solutions and innovations, there’s often one big elephant in the room: a lot of development projects don’t work. It is an uncomfortable and, I suspect for many, truly frustrating truth that hard efforts to combat poverty and increase food security often don’t pay off.
In 2011, the World Bank evaluated 86 projects and found that 41% had “non-positive outcomes.” Unfortunately, this is not an example that stands alone, and the trend understandably has project funders and investors keen to ensure that future projects deliver value for money.
At the same time, development projects keep getting more and more complex as realization grows that today’s challenges call for integrated solutions that can balance trade-offs and synergies between sectors and scales. So how can we, especially considering complexity and uncertainties, attain a better assessment of what might work and what might not?
A statistical model that can calculate return on investment
A recently published paper proposes a new, general framework that allows for calculating costs, returns and risks of development projects over their lifetimes. The framework is based on Bayesian network modeling, which can be used to predict probable outcomes of complex processes, especially ones with many unknown factors.
One helpful feature of Bayesian networks is the option to integrate expert knowledge with data, which is especially useful in places where data is scarce, such as in developing countries. Since it can also be costly to collect large amounts of data for individual projects, whereas expert knowledge is often more easily available, using Bayesian networks could prove a cost-effective way to assess development projects.
Decision makers can use the framework by entering values related to project budget, impact and risks into the model, expressed either as data ranges that represent their level of certainty about the values or as precise numbers. Once the model is populated, it can calculate the costs, benefits and return on investment over a certain period, while considering the given levels of uncertainty.
Because the model can be continuously updated, as new information becomes available, the authors suggest that it can be a powerful tool for project funders and implementers to evaluate projects’ current and likely future performance.
Considering and addressing uncertainties
The authors thus expect that the framework they propose can support decision makers through a development project’s lifetime. But, it’s worth highlighting their note on project planning: Because the model allows for detailing the degree of uncertainty of different risk factors, it also enables users to distinguish between important and less important unknowns. Making this distinction essentially allows implementers to adjust project designs to reduce potentially important risks and improve the chances of obtaining more favorable outcomes.
Research has shown that decision makers tend to overestimate the amount of data needed to make sound decisions, when in fact making strategic efforts to increase certainty on some, important factors could significantly improve the accuracy of probability assessments. Plans to expand the framework are underway, allowing decision makers to determine where to invest in a little more data to make the model a lot more useful for selecting and planning development projects with high likelihoods of success.
With such an array of potential benefits, you would expect development professionals to be lining up to put this project assessment framework into use, but so far uptake of the methodology has been slow.
Establishing new methods takes time
The researchers behind the framework note that the World Bank has expressed concern that the percentage of its projects justified by cost-benefit analysis – described as the most basic of project assessment tools – has been declining in the past several decades. Supposedly, this decline is in part due to the difficulty in applying cost-benefit analysis in situations where planners have little clarity on either the benefits or the costs of their projects.
Bayesian networks are well suited to address uncertainties about benefits and costs due to their ability to work without precise numbers and to incorporate expert knowledge. At present, however, most development professionals and researchers are unfamiliar with this methodology. Some are also uncomfortable with the reliance on expert assessments and the inclusion of cause-effect relationships that haven’t been confirmed in controlled experiments. It may therefore be a while before Bayesian networks are widely used in development planning.
Nonetheless, development professionals, and researchers stand to gain a lot from exploring the potential of Bayesian networks to provide practical decision support. These tools have been successful in solving practical problems and providing decision-relevant answers in many other disciplines, including computer science, public health, military operations, legal reasoning and natural resource management.
Bayesian networks can help development professionals figure out what will actually work, which in itself could provide a massive return for a comparatively small investment. Training decision analysts who could work as part of development teams could generate large returns in terms of increased impact and better use of development resources. The question that remains is whether the investors and donors, who are calling for stronger guarantees that projects will succeed, are willing to dedicate the time and funds required.