For almost two decades, the World Bank’s Doing Business index has stood as a shining example of the power of data to generate enabling environment reform. Doing Business scores, which measure the ease with which an entrepreneur can start and operate a business, are a common reference point for policymakers, investors and development partners in 190 countries, and the popularity of Doing Business has spawned many similar index-based approaches covering topics from agriculture to women’s economic empowerment.

So when news hit in August 2020 of allegations that staff had been pressured to manipulate the scores for China and Saudi Arabia, the implications reverberated through the development community, leading development partners and even the World Bank itself to ask the question: is Doing Business worth saving? Data irregularities were also identified for Azerbaijan and the United Arab Emirates (UAE).

The arguments in favor of Doing Business and similar indices are clear: what gets measured, gets done. By focusing on issues such as regulatory burdens and procedural delays faced by private businesses in a target country, often in potentially embarrassing fashion as compared to neighboring countries, Doing Business achieved what private interests previously could not — incentivizing governments to pass reforms that reduce the barriers to conducting business.

In response to the index, many countries have established a one-stop shop for business registration, a common Doing Business-inspired reform to reduce the time and cost of starting a business. In some countries, entire government agencies have been established to design and shepherd reforms to improve the country’s Doing Business scores.

The high visibility of the report leads countries to compete for the coveted position of top reformer. The competition for recognition extends far beyond just bragging rights; Doing Business scores have become a key criterion informing the decisions of foreign investors and development partners. For example, the Millennium Challenge Corporation (MCC) uses Doing Business scores as part of its country scorecards, which determine country eligibility for valuable development compacts.

Nonetheless, critics have long pointed to the risks of relying so heavily on an index that by necessity measures only a sliver of the regulatory environment. To enable comparison across countries, scores are based on very narrowly tailored business hypotheticals that do not reflect the diversity of experience across business size, location and sector. As such, Doing Business can unintentionally lead to cherry picking “quick wins” aimed at beating the scoring system over comprehensive reforms. Critics also argue that while spurring governments to act, Doing Business does so in a way that may prioritize foreign “best practices” over the voices of domestic citizens and local solutions.

An internal World Bank audit revealed in December 2020 that pressures on staff and actions of managers to override controls on data quality resulted in higher scores for China in 2018 and Saudi Arabia in 2020, which have since been corrected. (Data for Azerbaijan and the UAE were also corrected.) While these errors have been addressed, the incident provides a reminder of the danger of allowing a set of numeric scores to substitute for rigorous analysis of the complex political, economic, and social forces that impact market behavior.

Complementing Doing Business and other quantitative indices with deeper qualitative analysis provides the kind of detailed, context-specific research needed to move beyond numeric scores and toward local solutions. The Feed the Future Enabling Environment for Food Security (EEFS) project has designed, developed and/or advanced several qualitative diagnostic tools to do just this.

For example, EEFS recently developed the Inclusive, Entrepreneurial Market System Framework, which provides USAID and its implementing partners a detailed guide on how to assess both the formal and the informal institutions driving behavior within a market system. The results of conducting an IEMS analysis will provide deeper insights into why laws and regulations are incomplete or weakly enforced, and therefore how to fundamentally improve the environment for inclusive, entrepreneurial outcomes.

Another valuable qualitative tool is the Commercial, Legal, and Institutional Reform (CLIR) framework, which provides an in-depth view of the laws, regulations, institutions and social dynamics that impact the business environment. The CLIR methodology supports locally driven solutions by employing a final roundtable discussion that brings together public and private sector stakeholders to debate initial findings and discuss the way forward. The framework can also be tailored to specific areas of interest, such as agribusiness (AgCLIR), specific value chains (VcCLIR) or the seed sector (SeedCLIR).

Interestingly, the MCC now makes AgCLIR a standard part of its compact development due diligence for the agricultural sector to ensure its initial Constraints Analysis, based largely on datasets such as Doing Business, is complemented by thorough qualitative analysis. USAID also recognizes the importance of qualitative enabling environment analysis. In fact, USAID recently engaged EEFS to conduct a SeedCLIR in DRC, an AgCLIR in Ghana, and a VcCLIR in Mali to drive their evidence-based decision making in these countries.

Qualitative assessment approaches are also adaptable to changing conditions. In the past year, the travel restrictions brought on by COVID-19 have yielded opportunities to pilot new approaches with greater local ownership. For instance, assessments that pair international and local experts, conducting both virtual and on-the-ground interviews, present a cost-effective approach to designing solutions rooted in global lessons learned, which are adapted to fit the reality of a local context and help empower local experts.

Data-driven decision-making remains important for enabling environment diagnostics, and numeric scores still hold the power to identify gaps and encourage a call to action. But these scores must be examined, debated and put in context through qualitative analysis to ensure it is used rather than abused. Only then will we see Doing Business scores and those of other like indices applied as they should be — as a vehicle for cross-country knowledge sharing and an input into stakeholder dialogue.

The Feed the Future Enabling Environment for Food Security (EEFS) project is a pre-competed Blanket Purchase Agreement (BPA) for USAID Missions and Operating Units to access evidence-based analysis of how the enabling environment influences agricultural market system performance, food security and nutritional outcomes. For further information on how USAID can access EEFS expertise, please contact the Chief of Party, Adam Keatts, at akeatts@fintrac.com. For additional information on qualitative diagnostic tools for the enabling environment, please visit the Agrilinks page to check out the various resources we provide.