Critical reviewResearch writingEconomics & finance

The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business

How did a World Bank league table change what governments do? This review summarises the evidence in a well-known International Organization paper, tests its limits, and shows what a doctoral study of rankings, regulation and trade should do now that Doing Business has been discontinued.

What you will learn

  • Summarise the argument and evidence of Doshi, Kelley and Simmons (2019) accurately
  • Explain how global performance indicators influence states through reputation and comparison
  • Distinguish effects on regulatory behaviour from effects on trade or investment
  • Evaluate the validity problems of composite country rankings
  • Design an identification strategy for a dissertation that uses ranking data
  • Report a critical review of a single paper in a doctoral literature review

What you will unlearn

  • A rise in the Doing Business ranking shows that the business environment improved.The index measured formal rules in the largest business city, so a better rank shows changed rules on paper, not necessarily changed practice.
  • The paper shows that ranking reforms increased international trade.It shows effects on regulatory behaviour and investor perceptions; it does not estimate effects on trade flows.
  • Doing Business is still the standard source for business climate data.The World Bank discontinued the report in 2021, so new studies must justify historical use of the series or use its successor.
  • A correlation between rank and investment is evidence of impact.Countries that reform also differ in many other ways, so causal claims need a design that handles selection and timing.

Skills you will practise

  • Critical appraisal
  • Summarising a source faithfully
  • Measurement validity
  • Causal identification
  • Policy indicator analysis
  • Literature synthesis

Why review this paper

For about fifteen years the World Bank's Doing Business report ranked almost every economy on the ease of starting and running a firm. Governments set targets to climb it, donors used it to plan projects and investors read it. A doctoral researcher interested in rankings, regulation or trade will meet the index quickly, and will also meet a large grey literature that treats the rank as a direct measure of the business climate.

The most cited scholarly account of what the index actually did is Doshi et al. (2019). This review replaces it. It sets out what the paper claims, where its evidence is strong, where it is limited, and how a thesis should use it today.

Business rankings and international trade
Figure 1. Business rankings and international trade.

The paper under review

Table 1. Details of the reviewed source (Doshi et al., 2019).
ItemDetail
AuthorsRush Doshi, Judith G. Kelley and Beth A. Simmons
TitleThe power of ranking: The Ease of Doing Business indicator and global regulatory behavior
JournalInternational Organization, 73(3), 611–643 (2019)
ContextPart of a journal symposium on global performance indicators introduced by Kelley and Simmons (2019)
Evidence typesMarket share of business climate indicators; media analyses; observational data on state responses; an experiment with US portfolio managers; a case study of India
Main claimThe World Bank shaped business regulation worldwide through public ranking, without a formal regulatory mandate

What the authors argue and find

The starting puzzle is one of authority. The World Bank has no power to require any country to change its company law, yet many governments reformed business regulation in ways the index rewarded. The authors argue that the ranking itself was the mechanism. By grading states publicly and repeatedly, it created reputational pressure that officials and politicians felt and acted on (Doshi et al., 2019).

This argument follows the symposium's general account of global performance indicators. Kelley and Simmons (2019) propose that such indicators gain influence by framing an issue, by lending authority to their creators and by inviting repeated comparisons that make governments worry about their reputations. Doing Business is presented as a leading example.

The paper then assembles several kinds of evidence. It shows that the index held a dominant position among business climate indicators. Media analyses and observational data indicate that states responded to their rankings, and that some restructured their bureaucracies to manage the indicator, for example by creating units to coordinate reforms. An experiment with US portfolio managers tests whether rank information shifts investor sentiment, which would give governments a material reason to care. Finally, a case study of India traces a multiyear effort to climb the ranking, including the creation of rankings of Indian states (Doshi et al., 2019).

The authors' conclusion is that converging evidence from different methods shows how an international organisation can govern through information. In later work the same scholars describe rankings as "governance by other means": systems that inform audiences and spur competition among states rather than legislating directly (Kelley & Simmons, 2021).

Strengths

  • A clear mechanism. The paper explains why a ranking without legal force can change behaviour, and the explanation can be tested in other settings (Kelley & Simmons, 2019).
  • Triangulation. Media data, observational patterns, an experiment and a case study each have weaknesses, but they point in the same direction. A reader does not have to accept any single method alone.
  • An experimental test of the audience. Asking whether professional investors react to rank information addresses the step that the reputational argument most needs.
  • Attention to bureaucracy. Showing that governments built units to manage an indicator is concrete evidence that a ranking can reshape administration, not just rhetoric.

Limitations and methodological issues

What the index measured. Doing Business recorded formal rules, collected through questionnaires sent to local experts, and usually described the largest business city. It focused on the time and cost of compliance and assumed that firms know the procedures. The World Bank's own evaluators listed these features, noted that small data revisions could change rankings, and warned that frequent changes of method reduced the series' value for research (Independent Evaluation Group, 2021). Responses to the ranking may therefore be responses to the rules on paper, which is consistent with the paper's argument but limits any claim about the real business environment.

Selection into reform. Governments that chose to chase the ranking were not a random sample. Leadership changes, aid programmes and wider liberalisation could drive both reform and attention to the index. The evaluators also stressed that the report did not track the effects of reforms and that correlations do not justify causal claims (Independent Evaluation Group, 2021). Observational evidence of responsiveness is therefore suggestive rather than decisive.

From sentiment to behaviour. An experiment can show that rank information changes stated preferences. It cannot show how much the ranking moved actual capital flows, which is the outcome most trade and investment dissertations care about.

One deep case. India is an important case, but a country that made climbing the ranking a public goal is close to a best case for the theory. Cases where governments ignored the index would test it harder.

The index after 2021. In September 2021 the World Bank discontinued Doing Business after an investigation into data irregularities in recent editions, and it has since developed a successor assessment, Business Ready (B-READY). This does not undermine the paper's finding that the ranking was powerful. It strengthens a related concern: an indicator with that much influence attracts pressure on the people who produce it, a risk also documented for university rankings (Meho, 2025).

How the paper sits in the wider literature

The paper belongs to two literatures. In international relations it is a central study of global performance indicators, alongside the symposium introduction (Kelley & Simmons, 2019) and later theoretical work on rankings as regulatory systems (Kelley & Simmons, 2021). In economics, the index grew out of a programme that measured legal and regulatory institutions across countries and related them to economic outcomes; that programme is reviewed by La Porta et al. (2008), which a thesis should read to understand where the indicators came from and the critiques they attracted.

A third, useful comparison is the study of academic rankings. Management scholars have shown how rankings can distort the behaviour of the institutions they rank (Adler & Harzing, 2009), and recent bibliometric work documents anomalies consistent with gaming of university league tables (Meho, 2025). The same questions apply to country rankings: what is rewarded, what is neglected and who has an incentive to manipulate the numbers.

Finally, a thesis on rankings and trade must handle the trade literature carefully. A classic border-effect finding is sometimes misquoted as a comparison between Canadian "counties". The comparison was between trade among Canadian provinces and trade between provinces and US states of similar size and distance, and later work estimated much smaller effects. Read the original studies before citing a number from them.

Worked example: a design that tests the trade question (illustrative)

Suppose a master's or doctoral student wants to ask whether improving a country's rank increased its exports. The illustrative plan below shows how the paper's limits translate into design choices. No results are implied.

  1. Define the treatment precisely. Use the dates on which each edition was published and the size of each country's change in rank, rather than the rank level.
  2. Separate rule changes from rank changes. A rank can move because other countries changed or because the method changed (Independent Evaluation Group, 2021). Code these separately so the analysis can ask whether information or reform drives any effect.
  3. Choose an identification strategy. An event-study or difference-in-differences design compares countries before and after a rank change with similar countries whose rank did not change, and checks pre-trends (Angrist & Pischke, 2009).
  4. Test sensitivity. Report how results change with alternative controls, samples and time windows, so the conclusion does not depend on one specification (Leamer, 1983).
  5. State the scope. Results apply to the period of the index (to 2020) and to formal rules; say so plainly in the discussion.

What the paper can and cannot support

Table 2. Using Doshi, Kelley and Simmons (2019) in a literature review.
Claim in your thesisSupported by the paper?What else you need
The World Bank influenced regulatory reform through the rankingYes, with converging evidenceAcknowledge selection into reform
Governments created units to manage their rankYes, as reportedCountry examples from primary sources
Investors respond to rank informationPartly: stated responses in an experimentEvidence on actual flows
A better rank increased trade or investmentNoYour own causal design and trade data
The index measured the real business environmentNoMeasures of practice, such as firm surveys

Implications for doctoral researchers and common mistakes

Cite the paper for what it shows: that public ranking gave an international organisation real influence over national regulation. Use it to motivate questions about measurement, gaming and the politics of indicators. Do not cite it as evidence that ranking-driven reforms improved trade or growth. Related pages: a critical review of a citation ranking study, writing the literature review chapter and economic history in the economics PhD.

Common mistakes

  • Treating the Doing Business rank as a measure of how easy business really is.
  • Citing the index as current without saying it was discontinued in 2021.
  • Using rank levels across editions without checking for changes of method.
  • Inferring causation from a correlation between rank and investment.
  • Quoting famous trade findings second-hand, with the wrong units or comparison.
  • Relying on essay samples or summaries instead of the published paper.

Key takeaways

Your next step

Want to apply this to your own study?

  1. 1Tell us your topic, data and deadline
  2. 2A subject mentor reviews your work with you
  3. 3You get a plan and a quote
Literature review

References

  1. Adler, N. J., & Harzing, A.-W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education, 8(1), 72–95. https://doi.org/10.5465/amle.2009.37012181
  2. Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press. https://doi.org/10.1515/9781400829828
  3. Doshi, R., Kelley, J. G., & Simmons, B. A. (2019). The power of ranking: The Ease of Doing Business indicator and global regulatory behavior. International Organization, 73(3), 611–643. https://doi.org/10.1017/S0020818319000158
  4. Independent Evaluation Group. (2021). Approach paper: Doing Business indicators and country reforms. World Bank Group. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/ap_doingbusiness.pdf
  5. Kelley, J. G., & Simmons, B. A. (2019). Introduction: The power of global performance indicators. International Organization, 73(3), 491–510. https://doi.org/10.1017/S0020818319000146
  6. Kelley, J. G., & Simmons, B. A. (2021). Governance by other means: Rankings as regulatory systems. International Theory, 13(1), 169–178. https://doi.org/10.1017/S1752971920000457
  7. La Porta, R., Lopez-de-Silanes, F., & Shleifer, A. (2008). The economic consequences of legal origins. Journal of Economic Literature, 46(2), 285–332. https://doi.org/10.1257/jel.46.2.285
  8. Leamer, E. E. (1983). Let's take the con out of econometrics. American Economic Review, 73(1), 31–43. https://ideas.repec.org/a/aea/aecrev/v73y1983i1p31-43.html
  9. Meho, L. I. (2025). Gaming the metrics: Bibliometric anomalies in global university rankings and the research integrity risk index (RI2). Scientometrics, 130(11), 6683–6726. https://doi.org/10.1007/s11192-025-05480-2

How to cite this article

Choose your referencing style, then copy. Check your university's or journal's guide for local variations.

PhD Research Lab Academy. (2022, February 9). The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business. PhD Research Lab. https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade/

PhD Research Lab Academy. “The Power of Rankings: A Critical Review of Doshi, Kelley and Simmons (2019) After the End of Doing Business.” PhD Research Lab, 9 Feb. 2022, www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade. Accessed [access date].

PhD Research Lab Academy (2022) The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business. PhD Research Lab. Available at: https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade/ (Accessed: [access date]).

PhD Research Lab Academy. “The Power of Rankings: A Critical Review of Doshi, Kelley and Simmons (2019) After the End of Doing Business.” PhD Research Lab. February 9, 2022. https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade.

PhD Research Lab Academy, ‘The Power of Rankings: A Critical Review of Doshi, Kelley and Simmons (2019) After the End of Doing Business’ (PhD Research Lab, 9 February 2022) <https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade> accessed [access date].

PhD Research Lab Academy. The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business [Internet]. PhD Research Lab; 2022 Feb 9 [cited [access date]]. Available from: https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade/

PhD Research Lab Academy, “The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business,” PhD Research Lab, Feb. 9, 2022. [Online]. Available: https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade. [Accessed: [access date]].

@online{dissertation_on_effects_of_business_ranking_on_international_trade_2022,
  author = {{PhD Research Lab Academy}},
  title = {{The power of rankings: a critical review of Doshi, Kelley and Simmons (2019) after the end of Doing Business}},
  organization = {PhD Research Lab},
  date = {2022-02-09},
  url = {https://www.phdassistance.com/blog/dissertation-on-effects-of-business-ranking-on-international-trade},
  urldate = {[access date]}
}

This article is for learning. Use it to understand the method and cite the original sources above in your own work; do not copy or submit it as your own.

The power of rankingsNeed help applying it to your study?
Literature review