By Richard Smith, former editor BMJ.
[excerpt] We have long known that peer review [of medical research] is ineffective at detecting fraud, especially if the reviewers start, as most have until now, by assuming that the research is honestly reported.
I remember being part of a panel in the 1990s investigating one of Britain’s most outrageous cases of fraud, when the statistical reviewer of the study told us that he had found multiple problems with the study and only hoped that it was better done than it was reported. We asked if [he] had ever considered that the study might be fraudulent, and he told us that he hadn’t.
We have now reached a point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary.
Some supporting evidence comes from the trial having been registered and having ethics committee approval.
Andrew Grey, an associate professor of medicine at the University of Auckland, and others have developed a checklist with around 40 items that can be used as a screening tool for fraud (you can view the checklist here).
The checklist has been used to detect studies that have subsequently been retracted but hasn’t been through the full evaluation that you would expect for a clinical screening tool.
It may be time to move from assuming that research has been honestly conducted….to assuming it to be untrustworthy…
Stephen Lock, my predecessor as editor of The BMJ, became worried about research fraud in the 1980s, but people thought his concerns eccentric.
Research authorities insisted that fraud was rare, didn’t matter because science was self-correcting, and that no patients had suffered because of scientific fraud.
All those reasons for not taking research fraud seriously have proved to be false, and, 40 years on from Lock’s concerns, we are realising that the problem is huge, the system encourages fraud, and we have no adequate way to respond.
It may be time to move from assuming that research has been honestly conducted and reported to assuming it to be untrustworthy until there is some evidence to the contrary.
First published 5 July 2021, in the British Medial Journal, read the full report here.
Richard Smith was the editor of The BMJ until 2004.