Sarah V. O’Brien, BA, RN.
Consultant Nurse - Diabetes
Professional Nurse 2003 Jan;18(5):257-60.
Patients with diabetes account for 9% of total hospital costs and 8% of inpatient beds. They have poorer outcomes than other patients, as well as spending longer in hospital (Audit Commission, 2000; Donnan et al, 2000; Koproski et al, 1997; Davies et al, 1999).
These differences may be inherent to diabetes, but sub-optimal management in general wards may also contribute (Driskill, 1996; Hamilton et al, 2000). Patients lack confidence in general nurses’ and doctors’ ability to manage their diabetes (Callaghan and Williams, 1994), and nurses’ knowledge of diabetes is often inadequate (Leggett-Frazier and Vincent, 1994; Drass et al, 1989; Scheiderich et al, 1983).
Not least because of a predicted doubling in numbers of people with diabetes over the next 10 years (Audit Commission, 2000), these shortfalls in knowledge and management of diabetes by non-specialists need addressing. One standard in the diabetes National Service Framework (NSF) is to improve the management of inpatients with diabetes (DoH, 2001). If specialist diabetes teams are to provide ongoing education and support to health professionals, we need to be able to measure basic knowledge to evaluate interventions to improve it.
This paper reports on a questionnaire developed to test general nurses’ and trainee doctors’ knowledge of how to manage patients with diabetes in their wards, whatever their reason for admission.
Background to the study
A literature search of the Medline, Cinahl and Psyclit databases revealed few diabetes-specific knowledge questionnaires, and none from the UK. Some tools are in use but they are unpublished or have not been robustly validated.
Drass et al developed the most widely used diabetes knowledge questionnaire in the USA (1989), which is a modification of an earlier tool (Scheiderich et al, 1983).
All existing tools reviewed had been developed specifically for use with nurses (Baxley et al, 1997; Gossain et al, 1993; Jayne and Rankin, 1993; Leggett-Frazier and Vincent, 1994; Moriarty and Stephens, 1989). They had not been tested on doctors and did not meet our study’s requirements. Many tools had methodological weaknesses.
In particular, no study examined test re-test reliability of the questionnaires used. This is a measure of whether the questionnaire will produce similar results when administered at different time points to the same people. This is essential if the tool is going to be used before and after an intervention designed to improve knowledge. With no measure of test re-test reliability, results showing improved knowledge after a teaching programme would be unreliable.
We developed this new tool to measure nurses’ and doctors’ knowledge of how to manage diabetes because of the lack of UK-specific questionnaires, methodological problems with the existing tools and the lack of research undertaken among trainee doctors.
Research design and methods
Seven professionals from three hospitals in the north west of England were interviewed to determine what they expected ward based-nurses and trainee doctors to know about diabetes. They comprised five diabetes specialist nurses and two diabetologists.
The researchers developed the questionnaire on the basis of these interviews, a review of the literature on diabetes management, the US questionnaires and clinical experience.
The questionnaire contained 11 sections, each comprising six items. The sections were: physiology, blood glucose monitoring, medications, hypoglycaemia, insulin, hyperglycaemia, complications, diet, screening/prevention, surgery, and a general section. The questionnaire is not exhaustive, but covers the areas identified in the literature and interviews as most problematic in relation to the management of inpatients with diabetes. Respondents were asked to reply ‘Yes’, ‘No’, or ‘Don’t know’ to each question (Table 1).
Effort was made to ensure items were not ambiguous. The reading level was pitched at no higher than 12 years - generally accepted as good practice in questionnaire design (Steiner and Norman, 1995; Oppenheim, 1966).
Validity and reliability
Face and content validity establish whether the tool is appropriate for its intended use (Steiner and Norman, 1995). Face validity refers to whether a tool appears to measure what it should be measuring and content validity refers to whether the questionnaire has covered all relevant topics. To establish validity, a first draft of the questionnaire was sent out to four specialists - two diabetologists and two nurse specialists - and two general nurses from three hospitals, to assess its clarity, content and readability. Where necessary, some items were then reworded.
Reliability refers to how reproducible a test is (Oppenheim, 1966) - measurement at different times or by different people or with similar tests should produce the same results (Steiner and Norman, 1995).
For this study we measured internal reliability in nurses and trainee doctors using Cronbach’s alpha, and test re-test reliability, in nurses only, using Kappa statistic. The SPSS statistical package was used.
The higher the Cronbach’s alpha score (usually a range of 0 to 1), the greater the internal reliability of the tool. Cronbach’s alpha for the nurses was 0.81 and for the doctors 0.72, demonstrating acceptable internal reliability for both groups. Kappa statistic was also used, with a range from -1 to +1 and results about +0.60 are defined as indicative of good reliability. The Kappa coefficient was 0.689, indicating that the questionnaire has good stability over time. If scores were to increase following intervention, this should indicate a real improvement in knowledge (Litwin, 1995; Steiner and Norman, 1995).
The staff nurses in this study were of varying grades, from newly qualified to ward managers qualified for more than 10 years. All worked in general medical wards. Most nurses receive basic training in diabetes in their pre-registration courses and some may go on to do post-registration courses. None of the nurses in this study had done diabetes-specific training at a higher level.
The doctors were in their pre-registration year after completing their medical degree. Both groups managed patients with acute medical problems. Some of those with diabetes would have been admitted because of it; others might have had co-existing medical problems that complicated their diabetes during their hospital stay.
The questionnaire was sent to all 143 staff nurses in the medical unit in one general hospital and a convenience sample of 27 pre-registration house officers. (A convenience sample is a group that is easily accessible but may not be representative of the population as a whole.) The response rate was better for the doctors (100%) than the nurses (73%). The nurses were mailed their questionnaire because there were so many; the doctors filled in theirs in the presence of the investigator.
The number of nurses tested was much larger because within the hospital only a few trainee doctors are in post at any one time.
The junior doctors frequently rotate shifts and location, making it difficult to post questionnaires. Researchers asked them to fill in the questionnaire during a compulsory teaching session at which all the hospital’s trainee doctors should have been present.
Nurses and doctors’ responses were compared. Total scores from both groups were similar: doctors got an average 48 answers right out of 66 questions, and the nurses an average of 51 out of 66.
However, there were differences between the two professions in what they knew about some aspects of diabetes management. The questions relating to physiology and complications of diabetes were scored better by the doctors, whereas the nurses scored better on the questions relating to practical management of diabetes (Table 2). Both groups scored poorly on the questions relating to the timing of administration of insulin and whether to omit insulin when the patient’s blood glucose is low.
Both scored well on the sections relating to physiology of hypoglycaemia and hyperglycaemia. Both scored poorly on the questions relating to care of feet, management of insulin regimens during surgery, and driving and employment.
General nurses and trainee doctors have an important role to play in managing patients with diabetes. With the expected rise in the number of patients, and the long-term complications associated with it, all professionals involved in its management need an adequate knowledge base. A single tool to test diabetes knowledge accurately among nurses and doctors may be useful for future research examining diabetes management and strategies to improve it.
This study showed that knowledge of this field among general nurses and doctors is sub-optimal. Admission to hospital often signifies a period of instability for patients with diabetes. Regular screening for long-term complications and self-care skills are essential.
A hospital stay presents a chance to identify undiagnosed complications such as proteinuria and to reinforce patient education. But this cannot happen if, as these results suggest, ward staff lack the necessary knowledge.
This questionnaire took nurses and doctors only about 15-20 minutes to fill in, and could be used as an integral part of education programmes to improve their knowledge.
Following this study, we have developed an integrated care pathway at one hospital for the management of inpatients with diabetes. A randomised controlled trial to test the pathway has recently being completed. Patients admitted to the medical unit were randomised to either a care pathway for diabetes or to usual care. Results are being analysed.
A key endpoint of the randomised controlled trial will be the assessment of diabetes knowledge. The nurses tested for this study will be re-tested with the questionnaire to examine whether the pathway has improved knowledge. We intend to use this tool before and after diabetes education programmes to evaluate any change in professionals’ knowledge.
Currently, nurses and doctors are educated separately about diabetes management. The differences in knowledge among them suggest their combined knowledge would be complementary. A multidisciplinary programme of education and knowledge-sharing may be more effective than separate education programmes.
This study focused on hospital staff in general medical wards, where the nurses and trainee doctors had received no additional training to their pre-registration course. But it would be reasonable for the tool to be used with staff working in other areas, such as surgery. It would also be useful to test staff working in primary care and the community.
The NSF will encourage more diabetes care in these settings, and staff will need the necessary skills. Altering some of the questions for use with community-based nurses would not be difficult. This study’s response rates were good, indicating that people were willing to be tested. Many were keen to know how they had scored.
One limitation was the possibility of bias caused by the small number of trainee doctors. Another was the failure to examine test re-test reliability on these doctors. This will be necessary if we want to test the reliability of any interventions to improve doctors’ knowledge of diabetes. Although it would be difficult to track this particular group of doctors, internal reliability in junior doctors could be established by planning test and re-test dates to fit in with their rotation in the hospital.
Knowledge is only one factor influencing management of patients with diabetes. Future studies need to also consider causes of sub-optimal care in general wards.
Previous studies found knowledge of diabetes among general nurses was inadequate and this was confirmed by our questionnaire, which also identified a problem with trainee doctors.
This key area needs addressing to meet the standards of the diabetes framework. The fact that the combined knowledge of junior doctors and general nurses complements each other indicates that a joint approach to education might be beneficial. This questionnaire could become a useful tool for assessing the effectiveness of education programmes in the future.
- For copies of the questionnaire used in this study, write to: Sarah O’Brien, Diabetes Centre, Whiston Hospital, Prescot, Merseyside L35 5DR.
Audit Commission. (2000) Testing Times: A review of diabetes services in England and Wales. London: Audit Commission.
Baxley, S.G., Brown, S.T., Pokorny, M.E., Swanson, M.S. (1997)Perceived competence and actual level of knowledge of diabetes mellitus among nurses. Journal of Nursing Staff Development 13: 2, 93-99.
Callaghan, D., Williams, A. (1994)Living with diabetes: issues for nursing practice. Journal of Advanced Nursing 20: 132-138.
Davies, M., Currie, C., Dixon, S. et al. (1999)Evaluation of hospital diabetes specialist nursing (DSN): a prospective randomised controlled trial (abstract). Diabetic Medicine 16: A29.
Department of Health. (2001)National Service Framework for Diabetes: Standards. London: Department of Health.
Donnan, P.T., Leese, G.P., Morris, A.D. (2000)Hospitalizations for people with Type 1 and Type 2 diabetes compared with the nondiabetic population of Tayside, Scotland. Diabetes Care 23: 1774-1779.
Drass, J.A., Muir-Nash, J., Boykin, P.C. et al. (1989)Perceived and actual level of knowledge of diabetes mellitus among nurses. Diabetes Care 12: 351-356.
Driskill, W. (1996)Diabetes education: what every LPN should know. Journal of Practical Nursing 46: 1, 14-18.
Gossain, V.V., Bowman, K.A., Rovner, D.R. (1993)The actual and self-perceived knowledge of diabetes among staff nurses. Diabetes Educator 19: 215-219.
Hamilton, P., Everett, J., Kerr, D., Cavan, D.A. (2000)A ward-based clinical nurse advisor: effect on length of stay for patients with diabetes (abstract). Diabetic Medicine 17: (supp 1), A3.
Jayne, R.L., Rankin, S.H. (1993)Revisiting nurse knowledge about diabetes: an update and implications for practice. Diabetes Educator 19: 497-502.
Koproski, J., Pretto, Z., Poretsky, L. (1997)Effects of an intervention by a diabetes team in hospitalised patients with diabetes. Diabetes Care 20: 1553-1555.
Leggett-Frazier, N., Vincent, P.A. (1994)Measuring the diabetes knowledge of nurses in long-term care facilities. Diabetes Educator 20: 307-310.
Litwin, M.S. (1995)How To Measure Survey Reliability and Validity. London: Sage Publications.
Moriarty, D.R., Stephens, L.C. (1989)Factors that influence diabetic patient teaching performed by hospital staff nurses. Diabetes Educator 16: 31-35.
Oppenheim, A.N. (1966)Questionnaire Design and Attitude Measurement. London: Heinemann.
Scheiderich, S.D., Freibaum, C.N., Peterson, L.M. (1983)Registered nurses’ knowledge about diabetes mellitus. Diabetes Care 6: 57-61.
Steiner, D.L., Norman, G.R. (1995)Health Measurement Scales. Oxford: Oxford University Press.