Nursing research that has been withdrawn because of major errors or misconduct continues to be included in influential evidence reviews, according to a new study that warns patient safety may be at risk as a result.
Nurse researchers from Australia, who have published their findings in the International Journal of Nursing Studies, carried out a review of papers retracted from 116 leading nursing journals.
“The removal of a retracted trial from a review may substantially change the conclusions”
Despite the papers having been withdrawn, the new review revealed that some had gone on to be cited in systematic reviews that are used to help shape clinical practice.
According to the researchers from La Trobe University in Melbourne, academic papers were usually retracted “because of major errors or misconduct”, such as fabrication of data.
Yet, they found such potentially discredited or inaccurate findings were cited in reviews published after the papers had been retracted, raising questions about how that came to happen and the “integrity” of the later research.
“Nurses may be basing their practice on evidence that is wrong”
Lead author Richard Gray, professor of clinical nursing practice at La Trobe’s school of nursing and midwifery, warned there was a safety risk because nurses could be basing their practice on incorrect information.
“The inclusion of retracted trials in a systematic review might seem a bit dry and ‘academic’, but it represents an important risk to patient safety,” he told Nursing Times.
“The removal of a retracted trial from a review may substantially change the conclusions of that review. We have one example where an intervention is reported in the review as not working,” he said. “However, take out the retracted trial and it seems that the intervention does work.
“Nurses may be basing their practice on evidence that is wrong yet there is no way for nurses to know this,” he warned.
However, the study has sparked fierce debate in the nursing research sector with other academics suggesting it is unfair to question the integrity of review authors based on “very limited” data.
In all, the La Trobe team found 11 studies published in the 116 nursing journals from 2007 onwards had subsequently been retracted.
However, they also found these studies had subsequently been cited in a total of 37 different systematic reviews published in highly-regarded medical and nursing journals.
“Review authors need to demonstrate due diligence by checking the status of included papers”
In 14 cases, the citations were solely included for background or discussion and not part of the main review. But 23 systematic reviews included a retracted nursing trial as part of the main study.
Half were published before the trial was retracted. In another six instances it was not possible to say whether the review was published before or after the paper was withdrawn.
However, five reviews – citing four separate withdrawn studies – were published after those trials were formally retracted. The researchers said their findings raised questions about the “integrity” of these reviews and how retracted research had come to be included.
“It is important to consider how a manuscript that is clearly watermarked ‘retracted’ could be included in a systematic review,” stated the research letter that set out their findings.
The letter said: “It may be that review authors accessed manuscripts from a third-party website, such as ResearchGate or Academia, or had copies of manuscripts previously saved on their computer.
“We suggest review authors need to demonstrate due diligence by checking the status of included papers before they submit a review for publication,” it added.
“Nursing has the opportunity to take a lead on this issue and really improve the academic integrity of our research”
There were also questions for the editors of nursing journals when it came to the best way of dealing with the issue, said the letter. This could include publishing an addendum or retracting the review itself, it suggested.
Professor Gray told Nursing Times that it was unlikely that review authors had deliberately included retracted research.
“Most likely they made an error – albeit an avoidable error,” he said. “We argue that journals should require authors to check their references carefully – perhaps at the proofing stage – before the paper is published.”
He highlighted that he had written to around 25 editors responsible for the journals where the reviews were published and that he thought “they are taking the issue seriously”.
Ultimately, journal editors make the final decision over whether or not research should be retracted, usually in discussion with the authors.
“This issue affects medicine, allied health professionals and other clinical disciplines. Nursing has the opportunity to take a lead on this issue and really improve the academic integrity of our research,” he said.
Nurse research often ‘failing to adhere to trial best practice’
Professor Gray noted this was especially important given that the rate of retractions was increasing “exponentially” for reasons including plagiarism and duplicate publication.
However, he added that the retraction rate in nursing was still “extremely low and does not seem to be increasing at the same rate as in other disciplines”.
When it came to dealing with reviews that included retracted research, he said the response should be proportionate.
He said: “The key question is: Does the retraction of an included study significantly affect the conclusion of the review? If the answer is ‘no’, it may be appropriate to issue an erratum, perhaps stating that the review included a retracted study but the conclusions are not significantly affected.
“In some cases, the retraction of an included trial will impact the conclusions of the review. In this case there is a compelling argument that the review should be retracted,” he said. “This is harsh on the review authors, but a review that has unsound recommendations must be removed from the evidence base.”
However, Professor Gray told Nursing Times that he thought sanctioning authors was probably a step too far.
“Some review authors had clearly made a mistake including trials that had been retracted,” he said. “I think the task is to inform, educate and provide clear guidance to researchers about the need to check studies included in their review have been retracted.
“To my mind the inclusion of a retracted paper in a review is not research misconduct and sanctions against authors probably do not need to be applied,” he said. “Of course, there may be exceptions to this.”
However, a response to the La Trobe team’s study – also published in the International Journal of Nursing Studies – questioned their “sweeping recommendations”.
“Retraction does not guarantee that there are any issues with the study”
Dr Max Barnish, from the Institute of Health Research at the University of Exeter, said he was not convinced there was a big issue and was “concerned” by their conclusions.
“The numbers are very small and any claims of generalisability would appear far-fetched,” wrote Dr Barnish, who is a full-time systematic reviewer.
He argued that “publication status” need not necessarily preclude research being included in a systematic review under the established framework for carrying out this type of work.
“If publication status is not a criterion for inclusion, retracted studies are likely to be eligible,” he said, adding that just because a paper had been retracted did not always mean it was without worth.
“There could be good reasons to exclude retracted studies based on potential lack of trustworthiness, but this would depend on the accuracy of the decision to retract and what socio-political factors have influenced this,” he said.
“Retraction does not guarantee that there are any issues with the study, only that a human editor considered there might be,” he added.
Professor Gray has previously investigated the integrity of nurse research in terms of trials being registered, as reported last year by Nursing Times.
He found less than half of clinical trials published in leading nursing journals were officially registered, which he said raised serious questions about standards.