Risk assessment is often viewed as a key nursing responsibility, but does the evidence support its use in terms of delivering better care outcomes?
Peter Griffiths, PhD, RN, is director, National Nursing Research Unit, King’s College London; Andrew Jull, PhD, RN, is associate professor, School of Nursing, University of Auckland.
Griffiths P, Jull A (2010) How good is the evidence for using risk assessment to prevent pressure ulcers? Nursing Times; 106: 14, early online publication.
Nurses may assume that carrying out risk assessment and auditing such care processes result in better outcomes for patients. However, it is vital to examine the evidence on care processes before using these to demonstrate quality of care.
This article analyses the evidence supporting risk assessment for pressure ulcer prevention, which has been adopted as one of the eight high impact actions for nursing and midwifery. The issues raised are equally relevant to any clinical area where processes such as risk assessment are used to demonstrate evidence of quality care.
Keywords Tissue viability, Risk assessment, Pressure ulcer, High impact actions
- This article has been double blind peer reviewed
- Pressure ulcer prevention is one of the eight high impact actions for nursing and midwifery.
- Research evidence suggests some effective prevention strategies, such as frequent turning and the use of certain types of pressure relieving mattresses.
- Evidence does not support the role of pressure ulcer risk assessment tools in targeting prevention. There is strong evidence that they over predict risk and may lead to inefficient allocation of resources.
- If nurses use pressure ulcer risk assessment tools, they must combine this with clinical judgement.
- Quality pressure ulcer prevention is demonstrated by a low incidence rate, not the presence of a risk assessment tool.
- Similar issues arise when considering other high impact actions, such as falls prevention.
It is often assumed that risk assessment leads to better care outcomes. However, nurses must question and evaluate the quality of the evidence on which this assumption is based.
Pressure ulcer prevention has been adopted as one of the eight high impact actions for nursing and midwifery, which could lead to improved quality and reduced costs if implemented throughout the health service (NHS Institute for Innovation and Improvement, 2009). It is proposed as an indicator in acute care (Griffiths et al, 2008), in nursing homes (Nakrem et al, 2009)and may also be highly relevant for care in the community(NHS Institute for Innovation and Improvement, 2009).
Preventing pressure ulcers is seen to be a key quality indicator of nursing care and they are widely viewed as a “nursing sensitive outcome” (Van den Heede et al, 2007). These are the outcomes that are influenced by variations in the quality or quantity of nursing care received.
Underpinning the view that pressure ulcer prevention is a quality indicator is the belief that healthcare providers have the means and the tools with which to take effective action. This belief is based on the assumption that risk assessment tools accurately predict risk, that risk assessment is effective at reducing pressure ulcer incidence and that strategies to escalate management through increasingly complex pressure reduction devices will reduce incidence. These processes are often audited to determine the quality of care. Yet some studies have been unable to show a difference in pressure ulcer incidence between institutions where audits find good care processes and others with poorer performance (Bates-Jensen et al, 2003).
This article examines the evidence behind these assumptions to reveal how precarious they are. If the link between interventions and outcomes is uncertain, this in turn raises questions about how best to monitor and demonstrate the quality of nursing care. The questions asked here about the evidence to support indicators for pressure ulcer prevention should be asked of the evidence for indicators in any similar area where adherence to a process (such as risk assessment) is used as evidence of quality care. Other examples include falls and nutritional risk assessments.
Do risk assessment tools accurately predict risk?
Risk assessment tools are used to stratify patients likely to develop pressure ulcers into categories reflecting degrees of risk. First, we must consider whether the tools are accurate. However, the evidence suggests that none of the popular tools has good evidence for accuracy of their predictions.
Pressure ulcer guidelines suggest that risk assessment tools must be used alongside clinical judgement (Royal College of Nursing, 2005), principally because the tools do not accurately and reliably predict which patients are at risk. Studies on the Braden scale - the best researched tool – show a positive predictive validity of 4.5-100%, the Norton scale varies between 7.1% and 38% and the widely used but relatively untested Waterlow scale varies from 5.3% to 33% (Pancorbo-Hidalgo et al, 2006). This variability means that as few as 5% of patients identified by the most popular tools as being at risk of having a pressure ulcer go on to develop one (see Box 1).
Box 1. Accuracy of risk assessment scales
Accuracy of a positive result on commonly used risk assessment scales - the range of values found in research are:
Braden scale - 4.5-100%
Norton scale - 7.1-38%
Waterlow scale - 5.3-33%
Clinical judgement alone - 28.8-38.5%
Although evidence for the performance of the Braden scale is considerably more favourable than for the Waterlow score(Pancorbo-Hidalgo et al, 2006), the predictive validity of this tool may still be very poor. Predictive validity, however, is determined by incidence so the setting for the study may influence the performance of the tool (rather than the tool itself). The few studies of clinical judgement alone give positive predictive values ranging from 28.8% to 38.5%, although figures for sensitivity suggest that clinical judgement alone may miss a higher number of patients at risk than risk assessment tools do (Pancorbo-Hidalgo et al, 2006).
Admittedly, predictive validity may not be the best way to evaluate the effectiveness of risk assessment tools, as patients in validity studies are usually subject to pressure prevention strategies and so intrinsic risk is not being assessed (Defloor and Grypdonck, 2004). Predictive validity cannot be properly determined when using the tool can change the outcome – as it would be expected to if care changed as a result of a risk assessment. This would only be the case if the use of risk assessment tools led to effective preventative actions that reduce pressure ulcer incidence, but is there evidence that this is really the case?
Does using risk assessment tools prevent pressure ulcers?
Randomised trials of pressure ulcer prevention strategies are the most reliable means of assessing their effectiveness. However, a high quality systematic review found no randomised trials comparing use of a risk assessment tool with no formal risk assessment or comparing the performance of two different tools (Moore and Cowman, 2008).
A more recent study does provide some evidence indicating that using risk assessment may not in fact reduce the risk of developing a pressure ulcer. Vanderwee et al (2005) randomised 1,617 participants in six hospitals in Belgium to one of two conditions:
- A standard prevention strategy - a Braden score <17 or presence of non-blanching erythema (NBE) as indication of risk;
- A strategy of awaiting the presence of NBE to indicate risk.
If patients were assessed as being at risk using either approach they were assigned (randomly) to one of two types of pressure reduction devices (alternating pressure mattress or high specification foam constant pressure mattress), in addition to a standardised pressure management protocol.
The study found no overall significant difference in the incidence of pressure ulcers between groups according to the risk prediction strategy – 53 participants (6.7%) in the standard strategy group and 56 (6.8%) in the NBE only group developed pressure ulcers (European Pressure Ulcer Advisory Panel grades 2-4).
The investigators did find that the NBE-only approach was associated with significantly greater conversion to more severe grade 3 or 4 ulcers, which does suggest a possible benefit from the use of risk assessment tools. However, this conclusion was derived from a subgroup analysis. Such analyses can lead to false conclusions because of multiple significance testing(Oxman and Guyatt, 2002). Vanderwee et al (2005) reported 18 tests of significance on their dataset. When there is random allocation to groups there is a 1:20 probability of finding a significant difference by chance alone in each independent statistical test. As the number of tests rises, so does the probability of a “significant result”.
Fundamentally there is no way of properly determining from this study whether the significant result is simply the result of chance, as people are randomly allocated to groups, or because there is a real benefit from using the risk assessment tool. So what we have is, at best, suggestive evidence that is worthy of further investigation in a properly designed trial. Unfortunately, it does not constitute definitive evidence.
Vanderwee et al (2005) did however find that the use of pressure reduction devices in the standard approach was double that of the NBE only strategy (31.9% vs 15.5%). These findings suggest that not only do risk assessment tools overestimate risk, but also that they lead to much greater resource use – no doubt at considerable expense to healthcare organisations, given the substantial potential for overtreatment.
This is confirmed by other research. A large study of five acute care hospitals in the UK followed 2,507 patients in general medicine, general surgery, orthopaedics, urology, coronary care, acute care of older people, rehabilitation, and gynaecology between 1996 and 1998 (Clark et al, 2002). It found that overall 71% of patients were considered at some degree of risk using the Waterlow scale.
The scores from this risk assessment tool suggest that nearly three quarters of patients admitted to a general hospital may need pressure ulcer prevention strategies. This would be hugely expensive with no clear evidence that such a strategy would deliver benefit.
Do escalation strategies prevent pressure damage?
A more subtle use of risk assessment involves allocating patients to different risk categories. This might provide an answer to concerns about over treatment. Escalation strategies increase the frequency, intensity or complexity of interventions with increased levels of risk. For example, a clinical guideline might advocate moving “at risk” patients off standard mattresses and putting “very high risk” patients on alternating pressure mattresses (Royal College of Nursing, 2005).
Systematic reviews can provide estimates of the effects of different prevention strategies. Moving patients off standard hospital mattresses onto high specification foam mattresses may reduce the risk of developing a pressure ulcer by about 60%, while moving them from standard mattresses to alternating pressure mattresses may reduce the risk by about 70% (McInnes et al, 2008).
In the light of these small differences in estimates of effect it is less clear whether alternating pressure mattresses are superior to high specification foam mattresses and studies directly comparing the two approaches are needed. While there are trials comparing alternating pressure devices to foam overlays, these studies are small and of poor quality and no significant difference was found between the two types of device (McInnes et al, 2008). A large trial comparing an alternating pressure mattress with a high specification foam mattress (Vanderwee et al, 2005)found no significant difference in incidence of EPUAP grade 2 or greater pressure ulcers (15.3% vs 15.6%).
Unfortunately, the lack of a standard turning regimen on both mattresses means the effect of the different mattresses cannot be isolated and the equivalence in incidence rates may be because nurses took a different approach to turning patients on each mattress. The trial interventions may well have resembled common clinical practice, which involves maintaining a turning regimen on foam mattresses but not on alternating pressure mattresses. In areas where this difference in regimens is put into practice, it may not be necessary to use alternating pressure mattresses where turning and high specification foam mattresses can be used.
Healthcare providers could be wasting resources on the more expensive forms of pressure ulcer prevention, especially considering that the cost of alternating pressure mattresses is far higher than high specification foam mattresses or overlays(Nixon et al, 2006). However, once again there is an absence of definitive evidence, and a high quality trial is needed to address this gap in evidence on escalation strategies. An effective and apparently simple alternative escalation strategy is to increase the frequency of turning, which is effective in reducing pressure ulcer incidence (Defloor et al, 2005).
Unfortunately, the resources associated with turning may pose a barrier to its effective use. On a 25 bed general hospital ward, 18 patients would likely be “at risk”, “high risk” or “very high risk” using the Waterlow score (Clark et al, 2002), which would result in 216 turns per day on a two hourly turning regimen. Data is very limited, but assuming a range of 10-15 minutes per turn (Jaichandar et al, 2007; Hibbert et al, 1999), turning could require 36-54 nursing hours per day per ward if the Waterlow score was used to trigger escalation. The resource required to maintain such regimens may well be beyond the staffing capacity of wards.
The uncertainty of the evidence here is emphasised for two main reasons. First, it is striking that this area of care, which is so strongly “owned” by the nursing profession, urgently needs considerably more research. Ideally, data from an inception cohort (where patients are followed up from admission into the care setting) without risk reduction strategies is required to avoid the effect of such strategies on predictive accuracy. Use of these tools then needs to be evaluated in randomised trials against more declarative strategies, such as awaiting the development of NBE before initiating prevention strategies.
However, it is not ethical to implement “no risk reduction strategies”. The drive for quality measurement may reduce the likelihood that accurate tools can be developed, unless thoughtful research designs that strategically address the broad evidence gaps are funded. For instance, future trials testing existing risk assessment tools against a declarative strategy (waiting for the risk to declare itself by the presence of NBE) could also use the declarative arm as a validation cohort to develop better risk assessment tools.
There is a paucity of information on cost effectiveness, which would help to determine the best way to use limited financial and human resources to maximise effect. There is an urgent need to address the cost effectiveness of high specification foam mattresses in comparison to alternating pressure mattresses and the role of turning regimens. If the cheaper alternative is as effective, such evidence could deliver welcome cost relief for healthcare providers but the workforce implications need to be considered.
The second reason for emphasising uncertainty relates to how nurses are to assess and demonstrate quality. Measures of process have significant advantages as measures of quality when the process measured is known to be linked to an improved outcome. An example might be “door to needle time” for patients with a myocardial infarction, where it is clear that reducing the time between first presentation and intervention to unblock the coronary arteries reduces tissue damage and leads to improved outcomes. In the case of pressure ulcer prevention, as with many nursing interventions, it is hard to identify similar evidence based elements of care which, in themselves, are known to lead to improved care. The temptation is to select care processes that are easily measured even when, as in the case of the Waterlow score, there is no real evidence to support their use.
The incidence of pressure ulcers may well be a valid nurse sensitive outcome but preventative care processes are not good quality indicators. There is not enough research to clearly establish the role of any of the existing pressure ulcer risk prediction tools in delivering effective prevention (although there may be sufficient evidence to reject some).
Of course assessment on its own changes nothing but there is scant evidence that outcomes are improved by using a tool even when risk, as assessed by the tool, is linked to intervention. Given this limitation, it is hard to determine whether further preventative care based on the risk assessment is necessary or wasteful. It is certainly not sufficient to simply assume that the presence of a risk assessment represents quality care - unless action is taken. Further, the specific interventions that should result from an assessment of risk (with or without the use of a tool) are unclear.
It seems clear that the starting point for judging the quality of pressure ulcer prevention is to measure and report the incidence of ulcers. Simple reporting of rates will not allow meaningful comparison or benchmarking between wards and hospitals. Priorities for further research here include the development of a risk adjustment model that will allow the comparison of different provider units with different patient case mixes (Griffiths et al, 2008). But if we take the view asserted in the report on the high impact actions, that is, that pressure ulcers are “never events” any ulcers that develop represent a potential deviation from best quality care. The significance of an ulcer is clear, whereas the significance of a completed risk assessment tool is not.
More generally nurses should carefully consider the evidence for the link between care processes, such as risk assessment tools, and patient outcomes before deciding that a formal risk assessment should be adopted in practice and used as a marker of quality.
This brief overview of the evidence base for pressure ulcer prevention makes it clear that nursing quality cannot be easily determined by examining the specific preventative care used. That is not to say there is no information to guide practice in this area. Existing research and expert opinion do give strong pointers on effective preventative care, which involves clinical assessment (but not necessarily the use of “tools” to do so), moving patients off standard beds on to high specification foam mattresses, overlays or alternating pressure mattresses and the use of turning regimens in some circumstances.
Although there is probably sufficient evidence to give some confidence that actions and decisions typically within nurses’ responsibility are likely to make a difference, there is not enough to be prescriptive about what precisely should be done under what circumstances.
There is a clear need for better assessment tools that accurately predict risk in order to prevent overtreatment, since none of the existing ones seems adequate. The absence of good quality evidence does not necessarily mean that the use of risk assessment tools has no place. However, based on the limited available evidence it seems hard to justify using the presence of a completed Waterlow score as a marker of quality care and difficult to criticise practitioners who choose to use other and less formal approaches to assessment that might be superior.
The important thing is that practitioners do assess risk using their clinical judgements and, crucially, act on the assessments. Formal risk assessment tools may or may not have a place but the standard of care is defined by good clinical judgement and appropriate intervention, not the use of a tool. Ultimately it is shown by the absence of pressure ulcers.
- A version of this article appeared as an editorial in the International Journal of Nursing Studies (Jull and Griffiths, 2010). This work is informed by work undertaken by the National Nursing Research Unit, which receives support from the Policy Research Programme in the Department of Health. The views expressed here are not necessarily those of the DH.
Bates-Jensen BM et al (2003) The minimum data set pressure ulcer indicator: does it reflect differences in care processes related to pressure ulcer prevention and treatment in nursing homes? Journal of the American Geriatrics Society; 51: 1203-1212.
Clark M et al (2002) Collecting pressure ulcer prevention and management outcomes I. British Journal of Nursing; 11: 4, 230-238.
Defloor T et al (2005) The effect of various combinations of turning and pressure reducing devices on the incidence of pressure ulcers. International Journal of Nursing Studies; 42: 37-46.
Defloor T, Grypdonck MFH (2004) Validation of pressure ulcer risk assessment scales: a critique. Journal of Advanced Nursing; 48: 6, 613-621.
Griffiths P et al (2008) State of the Art Metrics for Nursing: a Rapid Appraisal. London: King’s College London.
Hibbert CL et al (1999) Cost considerations for the use of low-air-loss bed therapy in adult intensive care. Intensive and Critical Care Nursing; 15: 154-162.
Jaichandar KS et al (2007) A semi autonomous control and monitoring system for bed sores prevention. In: Proceedings of the 1stInternational Convention on Rehabilitation Engineering & Assistive Technology. ACM, Singapore, pp. 245-248.
Jull A, Griffithhs P (2010 Is pressure sore prevention a sensitive indicator of the quality of nursing care? A cautionary note. International Journal of Nursing Studies. 47; 5: 531-533.
McInnes E et al (2008) Support surfaces for pressure ulcer prevention. Cochrane Database of Systematic Reviews, 2008, Issue 2004. Art. No.: CD001735. DOI: 001710.001002/14651858.CD14001735.pub14651853.
Moore ZEH, Cowman S (2008) Risk assessment tools for the prevention of pressure ulcers. Cochrane Database of Systematic Reviews 2008, Issue 2003. Art. No.: CD006471. DOI: 006410.001002/14651858.CD14006471.pub14651852.
Nakrem S et al (2009) Nursing sensitive quality indicators for nursing home care: International review of literature, policy and practice. International Journal of Nursing Studies; 46: 6, 848-857.
NHS Institute for Innovation and Improvement (2009) High Impact Actions for Nursing and Midwifery. Coventry: NHS Institute for Inoovation and Improvement.
Nixon J et al (2006) Randomised, controlled trial of alternating pressure mattresses compared with alternating pressure overlays for the prevention of pressure ulcers: PRESSURE (pressure relieving support surfaces) trial. British Medical Journal; 332: 1413-1416.
Oxman A, Guyatt G (2002) When to believe a subgroup analysis. In: Guyatt G, Rennie D (eds.) Users’ Guides to the Medical Literature. Illinois, IL: AMA Press.
Pancorbo-Hidalgo PL et al (2006) Risk assessment scales for pressure ulcer prevention: a systematic review. Journal of Advanced Nursing; 54: 1, 94-110.
Royal College of Nursing (2005) The Management of Pressure Ulcers in Primary and Secondary Care: A Clinical Practice Guideline. London: RCN.
Van den Heede K et al (2007) International experts’ perspectives on the state of the nurse staffing and patient outcomes literature. Journal ofNursing Scholarship; 39: 4, 290-297.
Vanderwee K et al (2005) Effectiveness of alternating pressure air mattress for prevention of pressure ulcers. Age & Ageing; 34: 261-267.