Measuring impact is key to ensuring services are effective and efficient and to demonstrate the impact they have on patients and the wider society
With the publication of the Francis report, and last year’s publication of the national nursing strategy, it is clear that change is needed. Impact analysis is key to recognising, quantifying and implementing essential changes to the nursing profession. By calculating the value of the services nurses provide, we can see what works and how we can do better. Nurses are committed to improving patient care and impact analysis helps us to do this more efficiently.
Citation: Reed-Fox A (2013) Using impact analysis to measure healthcare. Nursing Times; 109: 24, 22-24.
Author: Angela Reed-Fox is a practice nurse at the Green Practice, Clevedon, and impact analyst at Fox Advising.
- This article has been double-blind peer reviewed
- Scroll down to read the article or download a print-friendly PDF including any tables and figures
The aims inherent in nursing are clear - promote and prolong health for as long as possible for as many people as possible. Demonstrating this impact of nursing is immensely powerful.
The Francis report (Francis, 2013) showed that simply hitting targets is not enough. A better way of analysing effectiveness is required if we are to consistently improve patient care, and this requires a more holistic analysis of actual outcomes for patients, not just indicators.
Impact analysis recognises an organisation’s impact is about more than the financial “bottom line”, it is about the social, economic and environmental impact, known as the “triple bottom line”. Analysis quantifies the value created by a service or project and increasing numbers of healthcare providers are recognising the need to measure the benefit they bring to patients and the wider society.
There are methods of impact analysis that can measure qualitative outcomes as well as quantitative targets and outputs. Often the outcomes that are difficult to quantify present the truest story of an organisation’s impact, particularly in healthcare, and measuring even something as intangible as improvement in quality of life is possible and a real indicator of impact. The reasons for carrying out impact analysis are outlined in Box 1.
Box 1. Reasons for impact analysis
- Provides evidence for clinical effectiveness and achieving aims
- Provides accountability and transparency
- Shows dedication to improvement and development of nursing services and policies
- Improves team morale
- It may be a legal requirement
- Helps to secure funding
The need for impact analysis
As budgets shrink and nurses are expected to do more with less, there is a need to ensure that the work they do is efficient, effective and has the required impact on patients. NHS England’s national nursing strategy envisages a nursing workforce providing a positive experience for patients and nurses, promoting independence and high-quality care. The proposed six action areas include impact measurement (Cummings and Bennett, 2012).
In 2012, the Public Service Act came into force. This requires all English and some Welsh bodies, including local authorities, NHS trusts, housing associations and fire services, to consider how the services they commission or procure may have a social, economic and environmental effect on the surrounding area (Social Enterprise UK, 2012). This is not a new idea - it has been used in the voluntary sector for many years - but healthcare has been slow to adopt it; now that budgets are being squeezed, more emphasis is being placed on effectiveness and efficiency.
Taylor and Bradbury-Jones (2011) remark on the change of focus in research with the emphasis on “what counts”. They suggest that, since nursing is a practical discipline and by its nature is committed to improving patient outcomes and promoting best practice, it lends itself to impact analysis; it also stands to benefit considerably as its contributions will be better recognised.
Measuring impact is long overdue; if we do not know what impact we are having on our patients, how do we know we are using resources effectively? As services are cut due to the economic climate, there is a danger of effective services being left unsupported and nursing roles being cut if impact is not measured. For example, specialist nurse teams are likely to have a great impact on patients’ quality of life and reduce reliance on doctors, but it is often these posts that are cut in times of economic hardship. The impact of these specialist teams should be analysed, as funding is more likely to be awarded to projects and teams that are achieving the most significant impact for their patients.
How to get impact-ready
Impact-readiness cannot be assumed when starting a project or new service. The aim of the project or the focus of the team needs to be discussed and agreed. At this point, it is ideal to involve patients or clients; as the proposed beneficiaries they will not only have opinions on what is needed, but also may have ideas on how it might be achieved. Involving patients from the start gives them “ownership” of the project or service and they are more likely to gain from it; it is also likely to improve the nurse/patient relationship.
The project can then be planned around the formulated aims and values, ensuring congruence within the team and transparency from the outset. For teamwork to be effective, it is essential that teams commit to a central goal regardless of the role they will play in achieving it.
After agreeing on the aim of the project or service, it is important to discuss how achievements will be measured. It is at this point that many people get confused between outputs and outcomes. Outputs do not necessarily measure the whole story, but outcomes demonstrate how the aim of the service has been achieved. For example, in a project supplying wigs to women undergoing chemotherapy, the aim is not to distribute as many wigs as possible, but to use wigs to improve quality of life through improved body image. In this situation, an example of an output could be “number of chemotherapy patients who used the wig service” whereas an outcome might be “improvement in body image from using the wig service”. Measuring the improvement in quality of life and body image demonstrates the project’s impact far better than the number of wigs supplied.
Next, the method of impact analysis needs to be decided on. Teams may opt to analyse impact in house or they may buy the services of an impact analyst. Every team and project is different. Sometimes it is more cost-effective to commission an analyst, particularly if the project is short term. Conversely, it might be more efficient to develop impact analysis as part of particular health professionals’ role for longer-term projects.
Impact analysis has been used in other settings for some time and there is a confusing number of tools available, all with distinct strengths and weaknesses. Some are simple and quick to use, and are suitable for inexpensive implementation in small projects; others are more comprehensive and are particularly useful for tenders. The focus of tools varies - some have an integrated client management system to demonstrate individual impact or distance travelled (how far a person has moved towards achieving a particular outcome, without necessarily achieving it outright - for example how a person as moved towards gaining a healthy weight), whereas others are more corporate and evaluate projects as a whole.
Particular care needs to be taken when choosing a tool; they are used in a variety of settings, work in different ways and measure different types of outcomes. It is useful, therefore, to discuss their efficiency with a team that has already used them or an impartial analyst not connected with particular tools. It is important to keep in mind that a tool suitable for one project may not be the best fit for another. Box 2 outlines a number of tools used in healthcare impact analysis.
Box 2. Examples of impact analysis models
Social Return on Investment
A reading group for mental health service users in a community centre could use the Social Return on Investment (SROI) tool. A ratio is calculated between what project costs and service user benefits; for example, every £1 invested, may yield £6.75 in benefit as perhaps service users experience a better quality of life, and increased confidence and independence. The SROI tool is rigorous but is not easy to personalise the results to particular client groups.
Ecco is a client management system that can be used in the management of long-term conditions. For example, to promote health among people with diabetes; weight, BMI and HbA1c can be inputted in the system (along with other results) as well as particular challenges patients face and their long- and short-term health goals and how they plan to achieve them. On subsequent visits, these subjects can be further discussed and changes logged. Patients can access the program from home as it is web-based (and secure). As a system, it promotes patient-centred practice, enabling patients to take control and see their progress. Anonymised results can be taken from the program as a means of demonstrating corporate as well as individual impact.
The Rickter Scale is quick to use and suitable for short interventions, such as a “know your blood pressure” event in a supermarket. It helps to scale individuals’ knowledge or concern about health issues.
Implementing impact analysis
When considering how impact analysis is to take place, it is also important to consider what will be required from team members. Impact analysis is essential in proving the significance of a team or project, but care must be taken that unfair demands are not put on team members. It is important that the whole team understands the importance of impact analysis, but it must not stand in the way of patient care or the aims of the project. Embedding impact analysis correctly will minimise disruption to the workforce, engage them in ownership of the end result and ensure that hard work is minimised as data is collected as the project progresses. Nurses understandably resent time spent in front of computers when they ought to be caring for patients, so care must be taken to ensure they do not spend time analysing when they should be nursing.
Impact analysis is about enabling teams to work more efficiently, so the process itself should be kept as lean as possible. Although they may be doing the analysis themselves, their unique relationship with patients makes nurses best placed to gather information from them. This should not require any extra time if nurses know what questions to ask. Conversations can take place between the patient and nurse while the treatment is happening and be fed back later.
Results of analysis
Information can be presented in a variety of ways; a cost to impact ratio is the simplest way of demonstrating the social/health impact compared with the cost of the project or service, but this does not provide the whole picture.
Adding colour to the story by using case studies aids understanding of the significance of the project. An impact analysis report should include an explanation of how information was gathered, justifying what was included and what was omitted. For example, a Social Return on Investment report might determine that a lymphoedema service yields £10 of value for every £1 invested. However, this figure should not be taken alone (as tempting as that might be), but should be part of the report, looking at what contributed to the service, what the outcomes were, and how this was indicated, as well as the narratives of service users.
Implementing change and next steps
The aim of impact analysis is to use the information gathered as a bid for funding or to demonstrate to stakeholders or the media the value of a project.
Each result will provoke questions - how could it have been done more effectively? Could the project have been done differently and had a greater impact, or had the same impact but by spending less money? How can efficiency be increased? Or how should efficiency be rewarded? Impact analysis provides a living document - it provokes questions and demands answers. The end result of every impact analysis is evaluation, which should always lead to a commitment to making changes, working more efficiently and achieving better results - similar to the nursing process.
Impact analysis shows the social, economic and environmental value of a service or project. When used properly, it is an efficient way of demonstrating how aims and objectives have been met, how lives or communities have been changed, and where improvements can be made to increase efficiency.
Impact analysis is a good measure of effectiveness and a driver for doing more with less. When embedded properly, it does not interrupt the activity of the team, but improves morale as successful working is recognised.
Healthcare has been late to catch up with impact analysis but, as efficiency and value for money is increasingly required, it is becoming indispensable. With recent changes in legislation and a growing requirement for measurable results in nursing strategy, it is a vital tool to demonstrate patient outcomes as well as efficiency and cost-effectiveness.
- Impact analysis assesses social, economic and environmental effects
- Embedding impact analysis into an organisation makes it easier to monitor performance and progress and adhere to founding values
- The Public Service Act means that all contractors have to demonstrate their social, economic and environmental impact in their area
- Rigorous impact analysis is a powerful way of attracting funding
- Impact analysis tools/methods should be embedded for maximum effectiveness
Cummings J, Bennett V (2012) Compassion in Practice: Nursing, Midwifery and Care Staff. Our Vision and Strategy. London: DH.
Francis R (2013) Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. London: Stationery Office.
Social Enterprise UK (2012) Public Services (Social Value) Act 2012: a Brief Guide. London: Social Enterprise UK.
Taylor J, Bradbury-Jones C (2011) International principles of social impact assessment: lessons for research? Journal of Research in Nursing; 16: 2, 133-145.