A look at whether administrative data in English hospitals contain enough information to reliably report “failure to rescue” rates and to what extent they affect patient safety
In this article…
- What “failure to rescue” (FTR) does and does not indicate
- How to judge whether FTR measurements are reliable
- Whether FTR rates can be derived from English hospital data
Keywords: Failure to rescue/Patient safety
- This article has been double-blind peer reviewed
- Figures and tables can be seen in the attached print-friendly PDF file of the complete article
KEY POINTS FOR POLICY
- There is clear potential to derive mortality-based failure to rescue (FTR) indicators for surgical patients from routine administrative data in England
- Our indicator, based on the United States’ Agency for Healthcare Research and Quality definition, is worthy of further exploration as a potentially valid safety measure
- FTR indicators may offer some advantages over standardised mortality measures (such as the Hospital Standardised Mortality Rate) for surgical patients and can, therefore, add to the range of indicators of hospital performance that are available
- Unadjusted FTR rates cannot be used to compare the quality of care between hospitals
- FTR does not appear to be a specifically nurse-sensitive indicator
Failure to rescue (FTR) refers to a death after a treatable complication. The rate of FTRs in surgical patients derived from routine administrative data is recognised as an important indicator of patient safety by the United States’ Agency for Healthcare Research and Quality. It holds the promise of being more sensitive to the quality of care in a hospital than either conventional mortality or complication rates (Silber et al, 2007).
In this Policy+ we draw on findings from a recent report from the National Nursing Research Unit to consider whether the FTR rate can be used in England (Jones et al, 2011).
What do FTR rates indicate?
Some deaths in hospital are inevitable. Much of the difference in death rates between hospitals has little to do with differences in the quality of care that people receive however, but is more related to the degree of illness and co-morbidity of patients who are receiving treatment, together with their level of vulnerability. Measures like the Hospital Standardised Mortality Rate try to account for this using statistical techniques, but no statistical adjustment can ever be perfect.
The rate of avoidable deaths among surgical patients with treatable complications - another way of referring to FTR - offers a different way of exploring how a hospital performs; this is one that relates to a specific group of patients and may offer some advantages.
For people undergoing surgery, the chance of developing a complication, like bleeding or pneumonia, is strongly related to factors such as their age or an underlying condition. Although complications are often a result of patient characteristics, however, the ability of a hospital to successfully treat a complication once it occurs is strongly related to the quality of care provided (Silber et al, 2007). The FTR indicator is intended to show how well hospitals perform once the complication occurs.
The potential significance of the FTR rate is reflected in recent reports and research, which emphasise the complexity of responding to patients who are deteriorating and highlight potential points of failure, including:
- Not taking observations;
- Not recording observations;
- Not recognising early signs of deterioration;
- Not communicating observations (Luettel et al, 2007).
For these reasons, FTR has often been considered particularly sensitive to the quality and quantity of nursing care that is available to patients (Griffiths et al, 2008).
How are FTR rates measured?
FTR rates used for both research purposes and as quality indicators are typically derived from hospital administrative databases. However, their validity can be compromised if the coding of secondary diagnoses (through which complications are identified) is poor; this is because the indicator relies on the identification of a group of patients who experience particular complications. Less-detailed coding of diagnoses means less chance that complications have been recorded. Previously, it has been concluded that English hospital data were unsuitable for deriving FTR measures, primarily because of low rates of coding (McKee et al, 1999).
In our study we aimed to assess whether the conclusions made about deriving FTR rates for surgical patients from English hospital data from 1996-97 still hold when using more recent data (until March 2009), and whether there was evidence that the rates derived were indicative of quality.
As a response to concerns about low rates of coding secondary diagnoses, we used alternative approaches that have been proposed, including:
- Assuming that all surgical deaths are FTR (Silver et al, 2007); and
- Using extended hospital stays that fall well outside the average range as an (indirect) indicator of FTR (Rafferty et al, 2007).
We used statistical techniques that allowed us to determine associations between FTR rates and a range of quality-related organisational factors derived from routine NHS data; these included staffing variables such as the numbers of nurses and doctors per bed, teaching status of the hospital and nursing staff stability.
What do the English hospital data reveal?
Our findings suggest that FTR indicators can be derived from English hospital data and that low levels of recording of secondary diagnoses are no longer a significant obstacle. We found:
- FTR rates are not significantly correlated with the average number of secondary diagnoses coded, which suggests an absence of systematic bias caused by coding practices;
- Strong year-on-year correlation for FTR rates in recent years, which suggests a degree of stability consistent with the rate reflecting an underlying characteristic of hospital performance;
- A number of associations between the FTR indicators and presumed markers of quality, some of which have been demonstrated in previous work in the United States (Jones et al, 2011).
However, not all the relationships we observed are clearly or plausibly indicative of variations in quality. It seems likely that there are confounding variables that were not accounted for in our models. Any FTR indicator would need to be adjusted for age and other patient-level factors before coming to a final judgement.
Non-risk adjusted FTR rates cannot be used to make comparisons between providers.
It is notable that previous work, which showed that higher nurse staffing was associated with lower levels of FTR, did not consider medical staffing (Silber et al, 2007). Although we observed a similar association when considering nursing in isolation, the association disappeared when we controlled for other factors. Higher levels of clinically qualified staff (doctors plus nurses) were associated with lower levels of FTR, and a higher nurse-doctor ratio was associated with higher rates of FTR.
Although our extended-stay FTR measure performed well in some respects, as there are no longer specific problems with secondary coding, there is no clear advantage for using this over the mortality-based measures. As an indicator of hospital performance, FTR is potentially linked to a number of factors beyond the control of the hospital, such as provision of community services. In addition, the absence of any association with the mortality-based indicators suggests that the measure based on the length of stay is not acting as a proxy of FTR as originally conceived.
Further work is required to develop and validate FTR as a quality indicator and develop and test risk-adjustment models.
This Policy+ was produced in collaboration with Professor Peter Griffiths at the University of Southampton, former director of the National Nursing Research Unit
Silber JH et al (2007) Failure-to-Rescue: comparing definitions to measure quality of care. Medical Care; 45: 10, 918-925.
Jones S et al (2011) An Assessment of Failure to Rescue Derived from Routine NHS data as a Nursing Sensitive Patient Safety Indicator. London: National Nursing Research Unit, King’s College London.
Luettel D et al (2007) Recognising and Responding Appropriately to Early Signs of Deterioration in Hospitalised Patients. London: National Patient Safety Agency.
Griffiths P et al (2008) State of the Art Metrics for Nursing: A Rapid Appraisal. London: King’s College London.
McKee M et al (1999) ‘Failure to rescue’ as a measure of quality of hospital care: the limitations of secondary diagnosis coding in English hospital data. Journal of Public Health; 21: 4, 453-458.
Rafferty AM et al (2007) Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional analysis of survey data and discharge records. International Journal of Nursing Studies; 44: 2, 175-1082.