The NHS should not use hospital-wide mortality rates as a “smoke alarm” to identify poor quality hospitals, a major study has concluded.
The research, commissioned by NHS England, found “no significant association” between hospital wide standardised mortality ratios and the proportion of avoidable deaths in a trust.
“We need to support and help hospitals to raise standards not criticise and punish them”
Researchers also said that producing “avoidable death rates” via case note reviews was not a good judge of quality, calling into question the basis of government plans to “band” hospitals according to their projected rate of avoidable deaths.
The review looked at the link between the two main mortality indicators and the proportion of avoidable deaths in a hospital as determined by case note reviews.
The hospital standardised mortality ratio and the summary hospital level mortality indicator measure whether the number of deaths observed at a hospital is higher or lower than expected, based on a statistical calculation.
In 2013, 14 trusts with higher than expected mortality rates were inspected by NHS England medical director Sir Bruce Keogh because of concerns they could be providing poor quality care.
Following the Keogh review, which placed 11 of the 14 trusts in special measures, Sir Bruce commissioned a study to look at whether standardised mortality ratios provide an accurate indication of the number of avoidable deaths.
- Early leaks from Keogh review point to nurse shortages
- Half of Keogh trusts had issues with raising of staff concerns
- Eleven trusts put in ‘special measures’ following Keogh review
The study, published in the British Medical Journal, was led by Dr Helen Hogan and Professor Nick Black from the London School of Hygiene and Tropical Medicine, and former health minister Lord Ara Darzi at Imperial College London.
Looking at 3,400 deaths in 34 randomly selected trusts, the largest ever nationally representative review of hospital deaths in England, the researchers found an overall proportion of avoidable deaths of 3.6%.
They found “no significant association between hospital-wide standardised mortality ratios and the proportion of avoidable deaths in a trust”.
Professor Black, a longstanding critic of mortality rates, said: “HSMRs can be distracting and potentially misleading and should not be used as a basis to praise or condemn a hospital or trust. We need to support and help hospitals to raise standards not criticise and punish them.
He added: “Given the complexity of hospitals – with many different clinical departments and activities – it is more helpful for the public, patients, staff and politicians to use a variety of specific measures of quality, such as adherence to good practice guidelines, outcomes for specific diseases or procedures, patient surveys of their experiences, infection rates and staff surveys.”
While the authors of the report concluded that hospital-wide mortality rates were not a good way of comparing trusts, they said looking at these indicators for individual, high fatality conditions could still be valuable.
Reviewing records of patients who die in trusts was also an “opportunity for identifying local quality problems and stimulating improvements”.
However, the review findings were challenged by Paul Aylin from the Dr Foster Unit at Imperial College, which has pioneered the use mortality rates – popularised by the “hospital guides” published each year in national newspapers.
Professor Aylin told Nursing Times’ sister title Health Service Journal that the BMJ study had a number of limitations, such as a small sample of case notes per hospital and flaws in case note reviews in attributing the preventability of deaths.
Meanwhile, Sir Bruce Keogh said the report’s findings provided “additional insight and show we need to look at a whole range of methods to measure healthcare and quality”.
Sir Mike Richards, the chief inspector of hospitals at the Care Quality Commission, said he agreed that “looking at mortality rates in isolation will not give a full picture of quality”.
“This is why we look at a range of measures during every inspection, including talking to staff and patients, to assess if a trust is providing safe and effective care,” he added.