Your browser is no longer supported

For the best possible experience using our website we recommend you upgrade to a newer version or another browser.

Your browser appears to have cookies disabled. For the best experience of this website, please enable cookies in your browser

We'll assume we have your consent to use cookies, for example so you won't need to log in each time you visit our site.
Learn more

The process of devising and undertaking a trial to evaluate oxygen therapy products

  • Comment

Sue Henshall, RGN.

Clinical Product Evaluation Nurse, Mid-Cheshire Hospitals NHS Trust

Clinical product evaluation involves reviewing hospital expenditure and usage of products to ensure that a trust is achieving value for money. Another driving factor is the need to promote the safe use of products and appliances as part of a clinical governance strategy.
Clinical product evaluation involves reviewing hospital expenditure and usage of products to ensure that a trust is achieving value for money. Another driving factor is the need to promote the safe use of products and appliances as part of a clinical governance strategy.

The main aim of my role of clinical product evaluation nurse in Mid-Cheshire Hospitals NHS Trust, is to standardise the top 100 spends throughout the trust with a view to reducing risk and improving safety.

This objective is facilitated through a clinical commodity advisory group, which meets once a month and, after a trial-and-evaluation process, decides which supplier and product has the quality to suit our purpose and is value for money. Our stated aims are: 'to improve patient, staff and stakeholder safety through the control of product purchasing and to advise on the purchase of equipment that demonstrates value for money, while meeting the need of the users.'

A range of products from several supplier companies is used across the trust, and it has been acknowledged that standardisation could result in significant savings. While it is anticipated that most of the savings would derive directly from competitive pricing, there would also be indirect savings in terms of a reduction of the risks associated with having several different products in use across departments.

The supply and control of medical products at the trust has changed in recent years. We no longer have a stores department on site and medical supplies are co-ordinated locally through a receipts and distribution department based at Runcorn. Deliveries are made throughout the week, with an individual cage for each ward and department. These are then distributed by the stores assistant from the receipts and distribution department.

The trust works closely with the NHS Purchasing and Supply Agency (PaSA) which, after consulting with trusts and specialist groups, decides which products to include in the Logistics catalogue. The PaSA website ( provides information for staff such as details of the contracts that have been set up and offers evaluation forms to use in product trials.

Selecting the products
Last year, our trust decided to review oxygen therapy consumables as part of our standardisation strategy. The trust's annual spend on oxygen masks is £35 000 - we use 46 000 masks, consumables, nebulisers and so on, a year.

Oxygen therapy consumables also encompass tubing, nebulisers and nasal cannulae. There are commitment discounts for oxygen therapy consumables purchased through NHS Logistics but in order to qualify a large commitment volume is required. At Mid-Cheshire the use of oxygen equipment is not great enough to qualify for any discount.

I selected three companies from the NHS Logistics catalogue to take part in the audit. The choice was based on price and quality: the need to save money was an issue but the prime motive was to ensure quality and obtain evidence to support the product choice.

The audit began with a trial of oxygen equipment in three areas: the medical admissions unit, A&E and a medical ward. Although the trial was deemed successful, in that the evaluation forms were completed, the advisory group concluded that a more comprehensive evaluation was required.

The questionnaire used in the trial had been simple - however, it had not used a scoring system. It had also invited general comments from staff, which had elicited both negative and positive remarks on the same product.

The inconclusive evidence of the trials led the advisory group to ask the three companies to make presentations to aid decision-making. A discussion of the advisory group that followed concluded that there was insufficient evidence to support any one company: the evaluation forms had been too basic and had failed to provide sufficient questions on issues such as fitting and connecting equipment. The consensus was that the situation needed to be re-evaluated. A new evaluation form that included a score rating was therefore devised.

Improving the evaluation
The new form was significantly more detailed and comprehensive than the initial form. It included more detailed information about labelling, ease of use, patient comfort and requested opinions on overall product quality. The group felt this was important in terms of providing a better overall picture and evidence to support any decision made. The new form enables calculation of a total score, which is then used to compare scores for products made by different companies.

The front sheet of the evaluation form - the product specification details and the package details - can be adapted for all products trialled; questions can be changed accordingly.

The oxygen equipment form included all the oxygen equipment consumables included in the trial - Venturi connectors, which control the volume of oxygen being supplied to patients, nebulisers, nasal cannulas and tubing.

Before the trial, a risk evaluation form was completed to determine whether it was safe to trial the oxygen therapy equipment. The forms were completed and signed by the supplies manager and me, then sent to our risk management team to be added to the risk register.

It is good management practice for all trusts to have a risk register as part of the Controls Assurance Standard and Clinical Negligence Standards (CNST) and Risk Pooling Standard Trusts (RPNST). The CNST and the RPNST are both run by the National Health and Safety Litigation Authority.

Once the new evaluation forms were finalised, a second trial was set up to compare same companies' products, but at different sites - these were theatres, a medical ward and the orthopaedic unit. The rationale behind switching to new sites was informed by the fact that medical and orthopaedic wards have housekeepers in post. It was felt that they would be better able to co-ordinate the trials as they have more time, leaving nurses with more time for patient care. Theatres were chosen because most of the oxygen equipment arrives with patients from the ward; patients are also sent back to the wards with oxygen equipment.

The use of the same equipment across departments was intended to remove the need to change equipment and reduce connection problems when patients moved from one area to another, such as from theatre to a ward - this was the main reason for wanting to standardise equipment across the trust. The orthopaedic unit was to be the main link area in the trials. The companies' products were trialled for one month, with the products provided free of charge. The companies' input into the trials varied: some simply provided the products, while others replaced all the products currently used with their own products; some provided personnel to help staff and provide training on using the product.

The completion of the final evaluation forms is now awaited. To date, 35 have been received, with a possible 10 outstanding. The data will then be collated and analysed by the clinical advisory group. The analysis will also include an estimate of the predicted use and expenditure for each company over a 12-month period. This will be presented on a spreadsheet for discussion by the advisory group, and to help predict the potential total trust spend for each of the suppliers. The results will be presented to the group, which will then make the final choice of products. Group members who were absent will be informed of the outcome in the minutes and will have the opportunity to question the outcome, if they wish, before the decision becomes final.

Implementation will follow discussion with the receipts and distribution supervisor. Dissemination of the trial outcome by the group members to their areas takes place and emails will be issued to all wards and departments by the supplies department, outlining the product changes and codes. Training by the chosen supplier will need to be arranged, particularly in departments that currently use products made by other manufacturers.

The use of the new equipment will be audited after six months to make sure that staff are happy with the products and that they have received sufficient training and resolved any problems. Checks will also be made to see whether any areas continue using suppliers other than the one selected.

The receipts and distribution department will then liaise with these departments to arrange the switch to the new products.

The trust, however, accepts that complete standardisation may not always be possible. Areas such as intensive care may need to retain some of their current equipment due to standardisation within the unit and compatibility with specialist equipment.

Guidance on setting up a clinical product evaluation trial
- Identify a need and draw up a specification for the equipment required

- Undertake a risk assessment

- Devise a product evaluation form. The inclusion of a ratings score on a range of criteria will aid comparison (see Box, page 49)

- Obtain financial approval/authority

- Approach more than one supplier - invite tenders

- Assess the products for overall value for money

- Award and manage the contract

- Choose the product, with reference to NHS Logistics

- Use national performance indicators to monitor and feed back data as required

- Share and disseminate information with regard to best practice

Products must be judged against an agreed national standard


- To follow a set of common protocols to enable users to make a fair comparison of products and to collect data in a format that will aid choice and ensure value for money

- To evaluate products throughout the trust in a uniform and unbiased way

- To provide a resource, to share information with users across the trust on product choices

- To evaluate new products

- To aid communication between product users and suppliers.

Criteria on which staff scored the equipment on trial
The trial involved medium concentration masks, high-concentration/non-rebreathing masks and equipment such as Venturi, nebulisers and tubing. The criteria scored included the following:

- Ease of set-up and positioning

- Efficiency of oxygen delivery

- Mask fit - it should fit securely and not easily be dislodged

- Patient comfort and ability to adapt equipment to individual patient needs

- Quality of strapping - it should secure mask while not chafing the patient

- Ease of connecting/disconnecting device and the speed with which this can be done (in the case of venturi/nebuliser/tubing)

- Ability to change nebuliser with Venturi

- Clarity of instructions for use and set-up of equipment.

The items were scored on a scale from 0 to 5. A comparison was made of the overall scores of products on test. The equipment on trial was also compared products already in use.

Author contact
Sue Henshall, Clinical Commodity Advisory Nurse, Mid-Cheshire Hospital Trust, Leighton Hospital, Middlewich Road, Crewe, Cheshire CW1 4QJ. Email:
  • Comment

Have your say

You must sign in to make a comment

Please remember that the submission of any material is governed by our Terms and Conditions and by submitting material you confirm your agreement to these Terms and Conditions. Links may be included in your comments but HTML is not permitted.

Related Jobs