The King’s Fund’s Point of Care programme held a workshop to examine various approaches to measuring patients’ experience
Jocelyn Cornwell, PhD, is director; Joanna Goodrich, MA, is senior researcher/programme manager; both at The Point of Care programme, The King’s Fund.
Cornwell, J., Goodrich, J. (2009) Exploring how to measure patients’ experience of care in hospital to improve services. Nursing Times;105: 29, early online publication.
With the new requirement that patients’ experience of care be measured as part of the drive to improve quality across the NHS, acute trusts face the challenge of choosing from a potentially dizzying array of options for carrying this out.
The Point of Care programme, which aims to improve patients’ experience of care in hospital and help staff to deliver the quality of care they would want for themselves and their own families, here explores its thinking on the topic.
Although many questions remain, we conclude that the most effective approach is likely to be one that incorporates both quantitative and qualitative information to give a more complete picture of the care pathway in individual trusts.
We encourage trusts – and in particular individuals responsible for and/or who want to improve patients’ experience – to draw on the range of measures to develop strategies that are most appropriate for their individual settings and needs.
Keywords: Patient experience, Measuring experience, Data collection
- This article has been double-blind peer-reviewed
Since 2002, the Department of Health has required data to be collected from millions of patients as part of the NHS patient survey programme. However, it was not until publication of High Quality Care for All (DH, 2008a) that patients’ experience was placed alongside patient safety and clinical effectiveness as integral to the government’s drive to improve quality in health care.
To ensure that quality improvement remains of high importance not only in policy but also in practice, the DH will:
- Require trusts to collect ‘real-time’ feedback to monitor patients’ experience and to publish quality accounts;
- Require that measures of patients’ experience be included in the ‘vital signs’ in the NHS operating framework and in the national indicator set;
- Reward high-quality performance through Commissioning for Quality and Innovation (CQUIN) (DH, 2008b);
- Provide guidance on gathering and using regular patient feedback (DH, 2009a; 2009b);
- Publish the results of national patient surveys on the NHS Choices website.
This new policy context creates an imperative to measure. The next step facing acute trusts is to address the question not of whether to measure but of how to do so while also articulating to themselves, their patients and staff why measurement is important.
There are many good reasons for measuring patients’ experience. Those that are of the most immediate concern to trusts are highlighted in bold in Box 1.
Box 1. Purposes of measurement
The purposes of measuring patients’ experience are to:
- Understand current problems in the way care is delivered;
- Inform continuous improvement and redesign of services;
- Help professionals reflect on their own and their team’s practice;
- Monitor the impact of any changes;
- Facilitate benchmarking between services/organisations;
- Compare organisations for performance assessment purposes;
- Inform referring clinicians about the quality of services;
- Inform commissioners about the quality of services;
- Inform patients about care pathways;
- Help patients choose high-quality providers;
- Enable public accountability.
Source: Coulter (2009, forthcoming)
What and how to measure
At the DH’s request, The NHS Information Centre for Health and Social Care (2009) has collated ‘indicators for quality improvement’, which will underpin quality improvement policy initiatives such as quality accounts. This comprehensive list of indicators has been drawn up based on those already used by healthcare organisations.
It includes five indicators that make explicit reference to nursing and numerous others in which nursing makes an important contribution (see Box 2).
It must be emphasised, however, that this is a rapidly changing policy area, and final guidance on what will need to be included in quality accounts has not yet been issued.
Box 2. Nursing indicators
- Score for patients who reported that nurses did not talk in front of them as if they were not there (PE39);
- Score for patients who reported that when they had important questions to ask a nurse, they always or sometimes were given answers they could understand (PE42);
- Score for patients who reported they always or sometimes had confidence and trust in the nurses treating them (PE43);
- Score for patients who reported that nurses always or sometimes washed or cleaned their hands before touching patients (PE54);
- Score for patients who reported that doctors or nurses gave their family or someone close to them all the information they needed to help care for them (PE24).
Examples of indicators to which nursing contributes:
- Score for patients who overall felt they were treated with respect and dignity while in hospital (PE37);
- Score for patients who reported they always or sometimes got enough help from staff to eat their meals (PE06);
- Score for patients who thought hospital staff did everything they could to help control their pain (PE09).
Source: The NHS Information Centre for Health and Social Care (2009)
Elsewhere, others are working on indicators that are sensitive to patients’ experience. The Picker Institute Europe, for example, has worked with Help the Aged to produce a set of indicators of dignity in care, based on research to find out what is important to older people (Magee et al, 2008).
In Northern Ireland, a collaboration of researchers at the University of Ulster and Belfast and South Eastern health and social care trusts have produced a set of key performance indicators for nursing and midwifery which are about the quality of nursing care – in other words, which should demonstrate the impact of nursing on patients’ experience (McCance and Telford, 2008). And in State of the Art Metrics of Nursing (Griffiths et al, 2008), the National Nursing Research Unit at King’s College London identified potential nurse-sensitive indicators.
Coulter (2009, forthcoming) describes a host of options that trusts might consider when developing their approaches for collecting data on patients’ experience of care. Table 1 shows the quantitative and qualitative methods currently available for collecting this data. For practical considerations relating to these methods and guidance on designing a measurement programme, see Coulter (2009, forthcoming).
Table 1. Methods for collecting data of patients’ experience
|Quantitative methods||Qualitative methods|
|Self-completion postal surveys||In-depth interviews, including video interviews|
|Interviewer-administered face-to-face surveys||Discovery interviews (Bridges et al, 2008)|
|Telephone surveys using live interviewers||Cognitive interviews (used in the early stages of developing questionnaires to protect against response biases)|
|Automated telephone surveys||Focus groups|
|Online surveys using web-based or email questionnaires||Web-based feedback|
|Surveys on hand-held devices||Comment cards, suggestion boxes|
|Surveys on touch-screen kiosks||Complaints and compliments|
|Surveys on bedside consoles||Patient diaries|
|Staff surveys||Mystery shopping and observation|
|Administrative data and routine statistics||Customer journey mapping|
Source: Adapted from Coulter (2009, forthcoming)
The Point of Care workshop
As the previous discussion shows, there are many different methods for collecting information; there is also, as we will explore more below, some uncertainty about which measures are the most useful.
While these are important issues to resolve, we should also note that measures will have little impact unless staff and managers make use of the information they reveal. Research for Seeing the Person in the Patient (Goodrich and Cornwell, 2008) revealed that, while many acute trusts were already doing plenty of data collection, in most cases the data collection and design of improvement strategies were segmented – that is, most trusts were failing to use the data to feed back into strategies to improve services.
Without such strategic data collection and use of the data, it seems unlikely there will be improvements in the quality of patients’ experience.
With the objectives of the programme in mind, and being aware that questions about what kinds of experience to measure and how to do measurement in this area were not easy, we organised a workshop on ‘Measuring patients’ experience of care’ in May 2009.
We wanted to explore, on one hand, the importance and value of measures to managers interested in service quality and patients’ experience. On the other hand, we also wanted to examine our concern that too much emphasis on measures and quantification can mean that fundamental aspects of human relationships are overlooked, including the experience of giving and receiving empathic and compassionate care. The latter is a particularly important point to be debated at another time.
The workshop was attended by: clinician managers responsible for improving and measuring patients’ experience; trust non-executive directors; operational managers; health service commentators and King’s Fund policy staff; and suppliers of web-based and other systems producing information for managers on patients’ experiences.
Participants at the workshop were asked to consider the following questions:
- In hospital, who needs information about patients’ experience of care? Why do they need it?
- What are the best methods and approaches for meeting these needs?
- What are the best or the most useful measures?
- What is the best or most useful way to feed back the findings on patients’ experience to individuals and groups at different levels of the organisation?
- What are the conditions required to motivate people to act to improve the experience of patients and families at different levels of the hospital system?
How to use measures to improve services
The consensus reached at the workshop is summarised in Box 3. The areas in which there remains considerable contention or where we thought there were broader risks worth articulating are discussed more fully below.
Box 3. Summary of findings
- Measurement of patients’ experience of care lags behind the systematic collection of data and use of measures to improve other aspects of quality. Hospital managers have for some time been systematically collecting data on the healthcare process (such as hospital waiting times, rates of infection and lengths of stay) and using these to improve services. Measures of patients’ experience should also be used to make changes.
- No single measure or approach to data collection will achieve all four of the purposes highlighted in Box 1. Trusts need to draw on both quantitative and qualitative information. Stories from both patients and staff remain crucial to giving us the complete picture of patients’ experience of care.
- Trusts should find out what information is already being collected and make use of this.
- Those trusts starting from scratch in designing measurement and improvement strategies would do well to collect simple data, and then gradually develop more sophisticated approaches to measurement.
For many of the questions put to workshop participants, there was agreement on some points with caveats attached (outlined below).
1. In hospital, who needs information about patients’ experience of care, and why?
Everyone needs to know about patients’ experience: patients and families; frontline clinical and support staff; non-executive and executive board members; and business, operational and clinical managers.
But individuals at different levels of the organisation need information for different purposes, and these different purposes may have an effect on the types of information they should receive (that is, qualitative versus quantitative) and how the information is shared with them.
2. What are the best methods and approaches for meeting these needs?
There is no one best method.
But those that are most useful are likely to be multifaceted in their approach, collecting both quantitative and qualitative data to give a fuller picture of the experience.
Over time, acute trusts should develop the capability to collect and make intelligent use of both types of information about patients’ experience for different purposes (see the case study for one such example of a trust making effective use of both the qualitative and quantitative data it collects).
3. What are the best or most useful measures?
It is not yet known which measures will be the most salient and useful in different clinical services. The DH’s current stance, rightly, is that it is too soon to opt for a standard set of measures of experience.
But we can conjecture that different measures will be useful for assessing different parts of the patient journey. And measures of patients’ experience should always take into account the experiences of staff as well.
4. What is the best or most useful way to feed back the findings on patients’ experience to individuals and groups at different levels of an organisation?
It is valuable to use data that already exists to identify quick wins, pinpointing areas for improvement that can be addressed immediately, so that measurement is seen to be effective and worthwhile.
But both in attempting to identify quick wins and in redressing more complex, deep-seated institutional problems, it should be noted that if the data on patients’ experience is fed back to staff who are not enabled to act on the information – that is, to actually do something to make changes and improve quality – measurement can actually backfire and result in poor morale.
Because of the natural defence mechanisms of individuals and organisations, data that reveals the need for improvement can place individuals, teams and whole hospitals in the position of having to defend themselves. This regrettably reinforces poor behaviours and faulty processes.
It is important to be constantly alert to the risk that people, teams and organisations will dismiss or ignore uncomfortable feedback, and to challenge such behaviour when it arises.
In addition, trusts should keep in mind that different ways of feeding back and analysing this information will be useful for different groups at different levels of the hospital. While high-level measures suitable for performance dashboards may be required at board level, they will not on their own give directors good insight into the experiences of patients and staff at clinical level. For this purpose, storytelling and direct observation are invaluable.
For frontline staff, quantitative data may be informative – and so too are qualitative reports and stories from ‘our own patients’.
5. What are the conditions required to motivate people to act to improve the experience of patients and families at different levels of the hospital system?
It is important to have a designated person within each trust to champion the agenda of improving patients’ experience of care.
But it remains unclear whether a single person designated for this role is enough to advance the agenda and how high up in an organisation this champion must be.
In addition, we need to motivate staff through appraisals, validation and internal training.
But we must ensure that appraisals and validation do not rely too heavily on quantitative measures and statistical indicators. Emphasis on patients’ experience and safety is welcome. However, if the pendulum swings too far to the other end of the spectrum – from an environment in which there is too little a focus on measures to too much of one – staff may very well come to view measures of patients’ experience as yet another box to tick.
Patient stories and other qualitative measures are among the tools most likely to re-motivate and enable staff to find ways to deliver high-quality, compassionate care, in spite of the many pressures of their jobs (Firth-Cozens and Cornwell, 2009).
Case study: ‘How are we doing?’ at King’s College Hospital
In 2004 King’s College Hospital in south London introduced its own patients’ survey – ‘How are we doing?’
The survey asks all inpatients, before they leave hospital, to give their opinion of the care they received, taking questions from the Care Quality Commission’s national inpatient survey, covering: patient engagement in care; the environment, including food; care perceptions; and demographic information. In addition, patients are asked for their comments and suggestions.
As a result, every month, good-quality information on patients’ experience is integrated into the regular statistics and performance indicators for the hospital (each month between 1,200 and 1,500 questionnaires are completed and around 700 comments are received).
The survey findings are discussed at a range of meetings at trust, division and ward levels – and action plans for improvement are produced. In 2009, the survey is being extended to day surgery and outpatients.
In December 2008, King’s went one stage further and began to produce a monthly patient experience report which analysed qualitative data, combining comments received in the ‘How are we doing?’ survey, together with complaints data, and the patient advice and liaison service (PALS).
The combination of quantitative and qualitative data provides a powerful picture of patients’ experience, built into the improvement cycle at King’s College Hospital.
Despite the complexity of measuring experiences of care and the questions that remain, the outlook for improving patients’ experience is promising.
Quite a lot of data has already been collected which can tell us about patients’ experience. We already know, for example, what matters to patients – see the Institute of Medicine’s (2001) dimensions of patient-centred care and Goodrich and Cornwell’s (2008) exploration of this.
As touched on above, we do have some useful indicators as well as a wide variety of qualitative and quantitative methods for collecting data that is worthwhile.
The important thing is that trusts commit to improving patients’ experience and begin not only using measures but also developing strategies to use the measures to improve services.
Bridges, J. et al (2008) Discovery interviews: a mechanism for user involvement. International Journal of Older People Nursing; 3: 206-210.
Coulter, A. (2009, forthcoming) Measuring Patients’ Experience of Care. London: The King’s Fund. Available from 3 August at: http://www.kingsfund.org.uk/pointofcare
Department of Health (2009a) The Engagement Cycle: a New Way of Thinking about Patient and Public Engagement (PPE) in World ClassCommissioning. London: DH.
Department of Health (2009b) Understanding What Matters: a Guide to Using Patient Feedback to Transform Services. London: DH.
Department of Health (2008a) High Quality Care for All – NHS Next Stage Review Final Report. London: DH.
Department of Health (2008b) Using the Commissioning for Quality and Innovation (CQUIN) Payment Framework. London: DH.
Firth-Cozens, J., Cornwell, J. (2009) Enabling Compassionate Care in Acute Hospital Services. London: The King’s Fund.
Goodrich, J., Cornwell, J. (2008) Seeing the Person in the Patient. London: The King’s Fund.
Griffiths, P. et al (2008) State of the Art Metrics for Nursing: A Rapid Appraisal. London: National Nursing Research Unit.
Institute of Medicine (2001) Crossing the Quality Chasm: a New Health System for the 21st Century. Washington, DC: National Academy Press.
Magee, H. et al (2008) Measuring Dignity in Care for Older People. London: Picker Institute Europe/Help the Aged.
McCance, T., Telford, L. (2008) Identifying Key Performance Indicators for Nursing and Midwifery Care: Report 1: The Consensus Phase. Ulster: University of Ulster, Belfast HSC Trust and South Eastern HSC Trust.
The NHS Information Centre for Health and Social Care (2009) Indicators for Quality Improvement. Leeds: NHS Information Centre.