One of the regular articles that the tabloid press keeps in reserve for use on otherwise slow news days is the good old favourite of the list of the professions that are the most despised by the general public. For some reason, we like to see whether most of our own personal hates are included in an official-looking list of the top ten most disliked professions, The usual suspects are always there, namely used car salesmen, lawyers, tax authorities, politicians, etc., but the list is not static — new members of the select list come and go. For example, a recent appointee is investment banking with its huge salaries and bonuses and need for massive bail-outs from most governments in the Western world. Another recent newcomer to the list is the “spin doctor” who appears to have an even worse rating than the politicians that they mostly serve. The contempt in which most spin doctors are held by the general public is understandable. The fact that their clear objective is to present, in the guise of apparently objective presentations of the underlying facts, a distorted, slanted or otherwise massaged view, (which of course benefits their own interests), is almost guaranteed to merit the scepticism and disdain of the public. The contrast with the medical/ scientific world in terms of the communication of new data and findings seems, on the face of it, to be huge. Here objectivity is the by-word — “massaging” the message has no place. The peer-review system by which any credible study is evaluated is there to root out and block any scientifically invalid publications. All we see are hard facts and valid conclusions drawn only from the underlying data.
And yet….., such a comforting view and the trust we all almost instinctively place in reported medical or scientific data may be misplaced. A study published in the May issue of the journal Radiology (Ochodo et al. Overinterpretation and misreporting of diagnostic accuracy studies; evidence of “spin”. Radiology 2013; 267 (2): 561 – 588) concludes that misreporting of results in diagnostic accuracy studies is rife, even in journals with high impact factors. What Ochodo and her team from Amsterdam, the Netherlands, did was to carry out a MEDLINE search of high impact journals (including radiology/medical imaging journals) for studies that reported on diagnostic accuracy studies. Articles included were primary studies of the accuracy of one or more tests in which the results were compared with a clinical reference standard. They then analysed a sample of these papers and found that as many as three out of ten of the papers contained “overinterpreted” results and a staggering 99% of the studies contained practices that would facilitate overinterpretation. In this context overinterpretation was defined as “the reporting of studies that distorts or misrepresents the results to make the intervention look favorable”. In other words, spin.
The most common form of actual overinterpretation was an overly optimistic abstract which contained stronger conclusions or test recommendations than those in the main text. This is particularly insidious since the abstract by its very nature is the entrance point to the complete paper. The content of the abstract is almost always the sole factor influencing whether the complete paper is worth reading or not. Other problems involved not stating the hypothesis, not reporting a confidence sample size calculation or not reporting confidence intervals. Although it’s not much of a consolation, the same rates of “spun” information were found in non-imaging and imaging journals. The main recommendations to curb the misreporting of results in diagnostic accuracy studies was that the publishing journals should insist that submitted manuscripts should conform to the “STARD statement” (STAndards for the Reporting of Diagnostic accuracy studies, www.stard-statement.org) Details of the STARD statement have been published in several prominent journals, including Radiology.
An additional action for the readers of the apparent findings in papers reporting diagnostic accuracy would be to act in the same way as with any spin — namely with a healthy dose of scepticism.
Alan Barclay, Ph.D. Editor, Diagnostic Imaging Europe (DI Europe) email: a.barclay@dieurope.com