The Road to False Positives: Evidence for Specification Search in Studies of Reg SHO

Bernard Black, Hemang Desai, Kate Litvak, Woongsun Yoo, and Jeff Jiewei Yu

Abstract

We provide evidence of specification search by re-examining results from 10 studies of an SEC randomized experiment (Reg SHO) that report evidence for a wide variety of indirect effects of the experiment, despite no evidence for a causal channel for these effects. Using either a simple, pre-specified research design or best-match specifications (which follow the sample and specification choices reported in the respective papers) we obtain null results, while reported results are mostly significant. This suggests that these papers made unreported specification choices, which produced significance. Evidence from Binomial and Kolmogorov-Smirnov tests is consistent with reported results reflecting specification search.

About the speaker

Professor Bernard Black is an internationally recognised scholar in the fields of law and economics, corporate governance, finance, and empirical methods for identifying causal relationships. At Northwestern University, he holds the title of Nicholas D. Chabraja Professor and is affiliated with both the Pritzker School of Law and the Department of Finance at the Kellogg School of Management. According to his official university profile, he has published more than 150 scholarly articles, and his work has been cited over 31,000 times on Google Scholar. He is the founding president of the Conference on Empirical Legal Studies and one of the founding editors of the Journal of Law, Finance and Accounting. Since 2010, he has also led summer research workshops at Northwestern University, demonstrating extensive experience in training researchers in empirical and methodological approaches.

Among his recent scientific contributions are the co-authored articles »The Effect of Disability Insurance Receipt on Mortality«, published in 2024 in the Journal of Public Economics, and »Non-Standard Errors«, published the same year in the Journal of Finance. These publications further enrich his substantial body of research at the intersection of economics, law, and public policy evaluation, while also demonstrating his continued active engagement in the international scientific community.

The host at the School of Economics and Business is Rok Spruk PhD.

We anticipate your attendance at the seminar with pleasure and appreciate your participation.