“What I’m alleging is that the reviewers, the editors of the so-called top-tier journals, grant review committees, promotion committees, and the scientific community repeatedly tolerate poor-quality science,” said Dr. Glenn Begley during a CrossFit Health event at CrossFit Headquarters on June 8, 2019.
Begley, CEO of BioCurate and a respected hematologist and oncologist, is famous for producing a study in 2012 that demonstrated most preclinical research related to cancer drugs cannot be reproduced. In this lecture, he shares the criteria he used for evaluating the research and the methods he found researchers used to distort their data. He then presents his diagnosis of the problem inherent in the academic research cycle, which he claims fuels the desperation that produces such distortions.
Begley’s six criteria for assessing preclinical research included the following questions:
- Were studies blinded?
- Were all results shown?
- Were experiments repeated?
- Were positive and negative controls shown?
- Were reagents validated?
- Was the analysis appropriate?
Through his analysis of research published in top-tier journals, Begley found that most data appeared to be distorted in order to produce a finding that the researchers had decided upon a priori.
“It’s easier to get the results you want if you know what you’re looking for,” Begley explains.
He highlights several methods the researchers he analyzed used to strategically present their data and exaggerate its significance. These methods included varying degrees of obfuscation, from selectively sharing results to representing cell growth as linear rather than exponential or logarithmic.
Begley refrains from characterizing such practices as outright fraud, instead opting to see them as symptoms of a systemic problem. “We get what we incentivize,” he says. The problem with scientific research, he argues, relates to problems with the academic research cycle and its promotion of perverse incentives that bear no relation to quality.
Despite his ominous portrayal of the state of laboratory science, Begley remains optimistic. “Science is the most effective tool humans have ever developed for driving progress,” he explains. “We’ve made enormous advances, but we could have done so much better,” he adds later.
To read a full transcription of the lecture, click here.
Dr. Glenn Begley: Perverse Incentives Promote Scientific Laziness, Exaggeration, and Desperation