Skip to main content
Feed the Future
This project is part of the U.S. Government's global hunger and food security initiative.
  • About
subscribe

Agrilinks

Search Log In

Main Menu

  • Topics
  • Events
  • Activities
  • Tools & Training
  • Members
  • About
subscribe

Limitations of Quantitative Scoring of the Enabling Environment: Is Doing Business Worth Saving?

profile image of EEFS Project profile image of Feed the Future Enabling Environment for Food Security

EEFS Project

Feed the Future Enabling Environment for Food Security

Jan 25, 2021
Export cargo area of airport
Export cargo area. Photo credit: Fintrac Inc.

For almost two decades, the World Bank’s Doing Business index has stood as a shining example of the power of data to generate enabling environment reform. Doing Business scores, which measure the ease with which an entrepreneur can start and operate a business, are a common reference point for policymakers, investors and development partners in 190 countries, and the popularity of Doing Business has spawned many similar index-based approaches covering topics from agriculture to women’s economic empowerment.

So when news hit in August 2020 of allegations that staff had been pressured to manipulate the scores for China and Saudi Arabia, the implications reverberated through the development community, leading development partners and even the World Bank itself to ask the question: is Doing Business worth saving? Data irregularities were also identified for Azerbaijan and the United Arab Emirates (UAE).

The arguments in favor of Doing Business and similar indices are clear: what gets measured, gets done. By focusing on issues such as regulatory burdens and procedural delays faced by private businesses in a target country, often in potentially embarrassing fashion as compared to neighboring countries, Doing Business achieved what private interests previously could not — incentivizing governments to pass reforms that reduce the barriers to conducting business.

In response to the index, many countries have established a one-stop shop for business registration, a common Doing Business-inspired reform to reduce the time and cost of starting a business. In some countries, entire government agencies have been established to design and shepherd reforms to improve the country’s Doing Business scores.

The high visibility of the report leads countries to compete for the coveted position of top reformer. The competition for recognition extends far beyond just bragging rights; Doing Business scores have become a key criterion informing the decisions of foreign investors and development partners. For example, the Millennium Challenge Corporation (MCC) uses Doing Business scores as part of its country scorecards, which determine country eligibility for valuable development compacts.

Nonetheless, critics have long pointed to the risks of relying so heavily on an index that by necessity measures only a sliver of the regulatory environment. To enable comparison across countries, scores are based on very narrowly tailored business hypotheticals that do not reflect the diversity of experience across business size, location and sector. As such, Doing Business can unintentionally lead to cherry picking “quick wins” aimed at beating the scoring system over comprehensive reforms. Critics also argue that while spurring governments to act, Doing Business does so in a way that may prioritize foreign “best practices” over the voices of domestic citizens and local solutions.

An internal World Bank audit revealed in December 2020 that pressures on staff and actions of managers to override controls on data quality resulted in higher scores for China in 2018 and Saudi Arabia in 2020, which have since been corrected. (Data for Azerbaijan and the UAE were also corrected.) While these errors have been addressed, the incident provides a reminder of the danger of allowing a set of numeric scores to substitute for rigorous analysis of the complex political, economic, and social forces that impact market behavior.

Complementing Doing Business and other quantitative indices with deeper qualitative analysis provides the kind of detailed, context-specific research needed to move beyond numeric scores and toward local solutions. The Feed the Future Enabling Environment for Food Security (EEFS) project has designed, developed and/or advanced several qualitative diagnostic tools to do just this.

For example, EEFS recently developed the Inclusive, Entrepreneurial Market System Framework, which provides USAID and its implementing partners a detailed guide on how to assess both the formal and the informal institutions driving behavior within a market system. The results of conducting an IEMS analysis will provide deeper insights into why laws and regulations are incomplete or weakly enforced, and therefore how to fundamentally improve the environment for inclusive, entrepreneurial outcomes.

Another valuable qualitative tool is the Commercial, Legal, and Institutional Reform (CLIR) framework, which provides an in-depth view of the laws, regulations, institutions and social dynamics that impact the business environment. The CLIR methodology supports locally driven solutions by employing a final roundtable discussion that brings together public and private sector stakeholders to debate initial findings and discuss the way forward. The framework can also be tailored to specific areas of interest, such as agribusiness (AgCLIR), specific value chains (VcCLIR) or the seed sector (SeedCLIR).

Interestingly, the MCC now makes AgCLIR a standard part of its compact development due diligence for the agricultural sector to ensure its initial Constraints Analysis, based largely on datasets such as Doing Business, is complemented by thorough qualitative analysis. USAID also recognizes the importance of qualitative enabling environment analysis. In fact, USAID recently engaged EEFS to conduct a SeedCLIR in DRC, an AgCLIR in Ghana, and a VcCLIR in Mali to drive their evidence-based decision making in these countries.

Qualitative assessment approaches are also adaptable to changing conditions. In the past year, the travel restrictions brought on by COVID-19 have yielded opportunities to pilot new approaches with greater local ownership. For instance, assessments that pair international and local experts, conducting both virtual and on-the-ground interviews, present a cost-effective approach to designing solutions rooted in global lessons learned, which are adapted to fit the reality of a local context and help empower local experts.

Data-driven decision-making remains important for enabling environment diagnostics, and numeric scores still hold the power to identify gaps and encourage a call to action. But these scores must be examined, debated and put in context through qualitative analysis to ensure it is used rather than abused. Only then will we see Doing Business scores and those of other like indices applied as they should be — as a vehicle for cross-country knowledge sharing and an input into stakeholder dialogue.

The Feed the Future Enabling Environment for Food Security (EEFS) project is a pre-competed Blanket Purchase Agreement (BPA) for USAID Missions and Operating Units to access evidence-based analysis of how the enabling environment influences agricultural market system performance, food security and nutritional outcomes. For further information on how USAID can access EEFS expertise, please contact the Chief of Party, Adam Keatts, at [email protected]. For additional information on qualitative diagnostic tools for the enabling environment, please visit the Agrilinks page to check out the various resources we provide.

Filed Under: Markets and Trade Policy and Governance

Comments

Part of the Activity

Feed the Future Enabling Environment for Food Security Project

Related Resources

Practical Analytical Framework for Inclusive Entrepreneurial Market Systems
The Role of Small-Group and Large-Group Dynamics in Market Systems

You May Also Like

post

A Practical Framework for Assessing Informal Norms in a Market System

Learn how applying the IEMS Framework can help USAID identify the underlying incentives for entrepreneurs and the factors that drive broad-based gains from investment to enable evidence-based decision-making.

profile image of EEFS Project profile image of Feed the Future Enabling Environment for Food Security

EEFS Project

Feed the Future Enabling Environment for Food Security

Nov 04, 2020
post

Spotlight on SeedCLIR: Incorporating a Maturity Model Approach to Seed Sector Development

Read up on SeedCLIR, a data-driven tool designed to enable countries to assess critical seed sector weaknesses, undertake targeted seed sector reforms and effectively gauge performance of reform activities over...

EEFS Project

May 11, 2018
post

Spotlight on AgCLIR: USAID’s Flagship Agriculture-Focused Legal and Regulatory Analytical Tool

In the third installment of the enabling environment analytical tools series, we take a closer look at how the AgCLIR tool offers a systems-based approach to identify root causes for...

EEFS Project

May 09, 2018
post

Measuring the Enabling Environment: What Works, and What’s Missing?

In the latest installment of its Feed the Future Learning Agenda series, EEFS explores the benefits and limitations of benchmarking practices for measuring the enabling environment for agricultural market systems.

EEFS Project

Feb 25, 2020
Follow Agrilinks:
  • Youtube
  • Facebook
  • Twitter
  • RSS
Agrilinks
Achieving agriculture-led food security through knowledge sharing
MarketLinks
Market-based solutions for development
LearningLab
A collaborative learning community of development professionals
Landlinks
USAID's knowledge sharing platform focused on land tenure and property rights
Climatelinks
A global knowledge portal for climate change & development practitioners
Urbanlinks
USAID’s sharing platform for resources on sustainable urban development
Resiliencelinks
A global knowledge platform for resilience practitioners
Globalwaters
Sharing knowledge and ideas to solve global water and sanitation challenges
U.S. Agency for International Development
  • Accessibility
  • Contact
  • Help
  • Privacy Policy
  • Terms of Use
The information provided on this website is not official U.S. government information and does not represent the views or positions of the U.S. Agency for International Development or the U.S. Government.

Search Agrilinks

My Account

  • My drafts and published content
  • My profile
Sign out

Contribute To Agrilinks

  • Add a post
  • Add an event
Need help?

Subscribe

Subscribe to receive our weekly newsletter featuring the latest and greatest Agrilinks content.

You'll also receive invitations to upcoming Agrilinks webinars and other special announcements. We won't share your email address and you can unsubscribe at any time.