Do you assess the care you provide to find out how good it is? A range of methods are available for you to check whether you really do provide high quality care
Naomi Reay, PhD, MA, DN, RSCN, RGN, is clinical commissioner/lead clinician (West Yorkshire), NHS Kirklees and Primary Care Research Network.
Reay N (2010) How to measure patient experience and outcomes to demonstrate quality in care. Nursing Times; 106: 7, early online publication.
Patients’ views on their care are an essential component in assessing quality of healthcare. A range of quality measures can be used, including patient reported experience measures and patient reported outcome measures. This one-part unit examines how to use questionnaires and other methods to measure both patients’ experiences and outcomes of care.
Keywords Patient experience, Outcome measures, PREMs, PROMs
- This article has been double-blind peer reviewed
1. Understand the importance of using patient reported experience and patient reported outcome measure assessments.
2. Know how to design and administer a questionnaire, and when other methods of data collection are appropriate.
All nurses aim to make patients better or help them cope with the reality of ill health, but how do they know if these outcomes are achieved? Patients and families can be generous with thanks and gifts, but we rarely have the opportunity to explore their experience or whether they have achieved what they wanted from healthcare interventions.
Several national initiatives, such as the model outlined in Supporting People with Long Term Conditions (Department of Health, 2009a), focus on personalised care plans and require nurses to focus on patients’ needs, recognising them as individuals with their own health and social circumstances. Therefore, if care plans are designed around individuals’ own needs, nurses need a way of knowing whether they have met those needs. This can be done by fully involving patients in developing and evaluating their own care plans.
Haywood (2006) stated: “Patients have an important role to play in communicating the impact of disease and the effectiveness of healthcare. Well-developed patient-reported outcome measures (PROMs) can provide a clinically relevant and scientifically rigorous resource for including the patient perspective in decisions about health care and subsequent evaluation.”
The NHS next stage review announced that the government would make payments to hospitals conditional on the quality of care patients receive. This would be based on a range of quality measures, including clinical outcomes, patient experience and patients’ views about the success of their treatment (DH, 2008).
The tools available to capture patient views are patient reported experience measures (PREMs) and patient reported outcome measures (PROMs). These are usually, but not always, in questionnaire form and should provide a rigorous system for collecting views across healthcare. Where questionnaires are not appropriate, other formats may be used, such as interviews or focus groups.
Nurses have a direct influence on patient experiences and outcomes of care, and may also be at the forefront of PREM and PROM collection. It is vital, therefore, that they understand what these measures are, how to use them and how the results they yield may influence wider funding and commissioning issues.
PREMs and PROMs
Questionnaires about people’s health status are not new; healthcare research has used PROMs (usually in questionnaire form) for many years as a means of seeking patients’ views on their health or the intervention being studied. PREMs are used to understand patients’ views on their experience while receiving care, rather than the outcome of that care.
Using information on both patient experience and outcomes enables us to have a broader understanding of service quality from patients’ viewpoint. It may be possible to have a service which provides good outcomes but a poor experience, or a good experience but poor outcomes. Our goal is to provide services which offer high levels of both.
Using PREMs/PROMs in clinical practice
Although PROMs have been used for many years, this has been patchy and the information from them has not necessarily influenced clinical care (Greenhalgh et al, 2005). The NHS next stage review (DH, 2008) has led to explicit requirements on the delivery of high quality care (DH, 2010), including involving patients and clinicians in planning and evaluating services. PREMs and PROMS are a key means of incorporating patient views into healthcare decision making.
The NHS contract for acute services includes the use of PROMs in four areas of planned surgery (DH, 2010):
- Varicose vein surgery;
- Hip replacement;
- Knee replacement;
- Groin hernia.
Designing patient centred questionnaires
PROMs and PREMs can provide information only on what patients are asked, and therefore the questions should reflect all the aspects that patients consider important. This means it is vital they are involved in developing the questions to ensure they are meaningful.
Even though many questionnaires appear to be a randomly gathered simple set of questions, key guidance (Medical Outcomes Trust, 2002)outlined how PROMs such as quality of life measures should be developed and used; these principles may be applied to the majority of questionnaires. They can be used to study a variety of aspects of patient outcomes, for example impairment, disability (activity limitation), participating in activities, quality of life or aspects of patients’ experience in using services.
While nurses are likely to be involved in administering PROMs and PREMs, it is also possible that they may be involved in selecting the questionnaire to be used. To ensure the one chosen is valid the stages of its development should be considered and evidenced, as these can vary greatly. Some have been developed by consulting patients about which questions to include, while others have used only groups of healthcare professionals to develop the questions and therefore their validity with patients is assumed rather than known.
Questionnaires use a variety of scoring methods. Some have scores that can be added to produce a number, some use labels such as “agree” or “disagree”, and some need computer packages for analysis and scores cannot simply be added up.
The people involved in carrying out PROMs should ensure careful planning, and as far as possible follow a protocol to ensure consistency in timing. For example, for a surgical PROM, there should be agreement about whether it is done pre-operatively and post-operatively and the timings such as “on the day” or at pre-admission clinic. Guidance on carrying out the four surgical PROMs required is available from the DH (2009b).
Patients should be given consistent guidance about completing the questionnaire. For instance, a question may ask: “Tell us about your symptoms in the past week”. Patients may ask whether they should think of their “bad” day during the week or the other “good” days; all staff involved must take a consistent approach. Finally, the methods of data entry and analysis should also be consistent.
This information should be agreed in advance, and guidance should be sought from those who developed the questionnaire on how to use and analyse the information obtained. PROMs are developed using clear processes, so it is extremely important that they are not altered in any way unless they are then retested for the key elements of rigour. These include:
- Validity - are the questions meaningful to patients, that is, are they answering the question we think they are?
- Reliability of the measure over time – can the questions give the same answer if nothing has changed for the patient?
- Translation (cultural and linguistic) - are versions available which have gone through the appropriate process for cultural and language translation? For example, a question for patients in the UK may read: “Can you do your housework?” For those in the US it may need cultural translation to: “Can you do your yard work or chores?”
Box 1 provides a case study on how PROMs can provide insight on the patient perspective. For more information on using these questionnaires, see tinyurl.com/oxford-scores.
Box 1. Case study
Amy Green*, a 72 year old woman, was struggling at home to climb stairs, walk outside, carry shopping and play with her grandchildren. She was also experiencing pain, caused by osteoarthritis in the knee, which was keeping her awake at night and making her tired and miserable during the day.
Mrs Green saw her GP, who referred her to an orthopaedic consultant. The consultant asked her to complete a questionnaire about her activities (PROM – Oxford knee score, see tinyurl.com/oxford-scores). This showed she was having problems with her activities of daily living at home and showed that pain was interfering with her ability to do what she wanted and keeping her awake at night. This, combined with clinical examination, meant Mrs Green was referred for a knee replacement.
She was asked to complete the PROM again at her preoperative assessment, and once more by her GP six months after surgery. This gave a picture of the impact of osteoarthritis on Mrs Green’s life before and after surgery, creating a clear picture of the outcome of the procedure from her perspective. She was also asked to complete a PREM questionnaire about her experience of being in hospital and the referral process from the GP to having surgery.
*The patient’s name has been changed.
Points to consider for PREMs/PROMs
Nurses involved in choosing a measure should consider the following:
- Is a questionnaire format appropriate for the patient population and the information you need to gather?
- What outcome/s or experiences are you trying to understand?
- What measures are already available which sound appropriate from their title?
- From the original article on the measure (describing its development), can you establish the following:
- What is the questionnaire measuring?
- What is the conceptual basis? For example, there are many approaches to quality of life (QoL) - is the questionnaire based on health related QoL (HRQoL) or health utilities such as quality adjusted life years (QALYs) and so on? These are complex concepts; in brief, HRQoL explores domains relating to health such as impairment and activity limitation. QALYs require patients to rate goals in relation to risk taking and hypothetical states. More detail on these concepts and others can be found in Rapley (2003).
- Is there clear evidence that patients have been involved in identifying the questions in the PREM/PROM, or do your patients feel the questions cover their areas of interest?
- Is there clear evidence of rigour in the design including: validity (is it asking questions which are relevant to patients?); reliability (has it been shown to be reliable over time?); and, if it is available in other languages, is there evidence of how these translations have been verified?
Formats for patient responses
PREM/PROM questionnaires usually ask patients to answer the questions using one of a range of possible formats. The aim is to provide a wide enough range to give patients a choice of responses but not so much choice that it becomes difficult to differentiate between each option.
Two answer formats commonly used are Likert scales and visual analogue scales.
Likert scales are a rating scale (Henerson et al, 1987; Likert, 1932) in which patients are given statements and asked to rate their response. A common form gives a range of five potential answers to choose from, which may include a range from “strongly disagree”, to “strongly agree”. Fig 1 shows an example of a Likert scale.
As with other questionnaire formats, those designing the questions are recommended to follow a series of stages when drawing up the statements provided for respondents (Henerson et al, 1987).
Visual analogue scales
A visual analogue scale (VAS) is a popular choice of response format for a number of reasons. It offers a range of responses and uses language to describe the parameters (otherwise described as anchor points) of whatever is being measured (none at one end, worst possible at the other end).
Since it should also be a line of a known length - usually 10cm (Cline et al, 1992), it is possible to produce a numerical score by measuring the point where respondents mark the line from the bottom or left point (usually the lowest level of the phenomenon being measured). This gives a measure in centimetres or millimetres relating to the mark on the line. Fig 2 shows a vertical VAS.
A VAS can be used to represent many symptoms or states, for example pain or emotions. They have been used in research for many years and information on their design has been gathered. The vertical design is considered preferable (Waltz et al, 1991, cited in Parahoo, 1997), as it gives a natural continuum between no symptoms at the base, to the most at the top and avoids potential error in measurement, described as problems with “left-right discrimination”. This is where respondents’ marks on a horizontal line are measured from the right instead of the left, for example giving a mark of 3/10 instead of 7/10 (Waltz et al, 1991).
Although many PREMs/PROMs are carried out as questionnaires, this format is not always the best choice for all patient groups.
For example, if we consider the ageing population and the drive to deliver care closer to home, there are particular issues for those in the community who may be housebound and therefore not able to post a questionnaire. Also, for those who cannot read or write, a postal questionnaire may not be feasible.
In these situations other formats can be used, including qualitative methods of data collection such as interviews or focus groups. Methodological techniques need to be considered when using these; the temptation to sit with people and read out a questionnaire should be resisted.
There are several types of interview (Parahoo, 1997). Three of the most commonly used are: structured (using a list of pre-determined questions in order); semi-structured (interviews based on a pre-determined topic list); and unstructured (without pre-determined questions or topic list). An advantage of using interviews is that they may be able to gain an in depth understanding of a patient’s outcome or experience and explore issues important to them. They do, however, take time and require skill to plan, conduct and analyse.
Focus groups offer an opportunity for interaction between participants which, if well facilitated, may stimulate discussion (Bloor et al, 2002).
Ground rules should be developed and agreed with participants, such as confidentiality, recording, what happens if participants want to end it or if they become distressed (Holloway and Wheeler, 2002).
PREMs and PROMs are becoming increasingly important in the NHS, as they aim to provide insight into patients’ views of their experiences and outcomes. This insight can help nurses to understand the impact of their care and review their practice accordingly.
Many PREMs and PROMs take the form of a questionnaire but it is possible, and sometimes more appropriate, to use other formats such as interviews and focus groups.
Since patients’ views on quality are a vital part of the information that will be used by healthcare commissioners, it is vital that nurses use PREMs and PROMs appropriately. Involving patients in developing and selecting these tools is crucial to ensure we ask the right questions in the right way.
For the Portfolio Pages corresponding to this unit see the document above.
Portfolio Pages contain activities that correspond to the learning objectives in the unit. They are presented in a convenient format for you to print out or work through on screen and can be filed in your professional portfolio as evidence of your learning and professional development.
Click here for more Guided learning units.
Bloor M et al (2002) Focus Groups in Social Research. London: Sage publications.
Cline M et al (1992) Standardization of the Visual Analogue Scale. Nursing Research; 41: 6, 378-80.
Department of Health (2010) Guidance on the NHS Standard Contract for Acute Services 2010/11. London: DH.
Department of Health (2009a) Supporting People with Long Term Conditions: Commissioning Personalised Care Planning – A Guide forCommissioners. London: DH.
Department of Health (2009b) Guidance on the Routine Collection of Patient Reported Outcome Measures (PROMs). London: DH.
Department of Health (2008) High Quality Care for All. NHS Next Stage Review Final Report. London: DH.
Greehalgh JA et al (2005) The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? SocialScience and Medicine; 60: 4, 833-843.
Haywood K (2006) Patient-reported outcome 1: Measuring what matters in musculoskeletal care. Musculoskeletal Care; 4: 4, 187-203.
Henerson ME et al (1987) How to Measure Attitudes. California: Sage.
Holloway I, Wheeler S (2002) Qualitative Research in Nursing. Oxford: Blackwell Publishing.
Likert R (1932) A Technique for the Measurement of Attitudes. New York: Columbia University Press.
Medical Outcomes Trust (2002) Assessing health status and quality of life instruments: attributes and review criteria. Quality of Life Research; 11: 193-205.
Parahoo K (1997) Nursing Research. Principles, Process and Issues. London: Macmillan.
Rapley M(2003) Quality of Life Research – a Critical Introduction. London: Sage Publications.
Waltz C et al (1991) Measurement in Nursing Research. Philadelphia: FA Davis.