Understanding and critiquing qualitative research papers
The first article in this series on understanding research (Lee, 2006a) examined the basic terminology used by researchers and identified that qualitative research produced non-numerical (qualitative) data. This type of research aims to report a situation as it actually is in a natural rather than a laboratory setting.
Qualitative researchers justify this approach by suggesting that it is not possible to separate the context or setting in which the phenomenon occurs from the phenomenon itself (Morse and Field, 1996).
VOL: 102, ISSUE: 29, PAGE NO: 30
Polly Lee, MSc, BA, RSCN, RGN, RM, DipN, ILTM, is lecturer in child health nursing, City University, London
Understanding qualitative research
Some of the terminology that relates to qualitative and quantitative research and how these relate to different worldviews (paradigms) was introduced in the first article in this series. The notion of qualitative (non-numerical) and quantitative (numerical) data was also introduced. Readers of qualitative research need a sound understanding of the terminology specific to this type of research to make full sense of the report.
Within qualitative research there are different approaches (methodologies). These may be more easily understood by returning to the example used in the first article of the series. This discussed a patient being asked about their experiences of receiving different treatment for hypertension.
A researcher undertaking a study on this topic may: follow a particular ethnic group over a prolonged period of time - in the past this would have involved living among that population (this is known as ethnography); explore what patients understand about the different treatments and build a theory as the research progresses (grounded theory); or explore the lived experience of hypertension (phenomenology). These are the most common approaches used in qualitative research.
There are several frameworks for critiquing research, some of which have been constructed to critique or evaluate both qualitative and quantitative studies. Others, however, have been constructed to critique only one of these. Box 1 shows some of the commonly utilised frameworks within the nursing literature.
This article focuses on one framework designed for critiquing research. The Critical Appraisal Skills Programme (CASP) framework has been chosen as it has separate frameworks for qualitative and quantitative research. It is also available on the internet, with more detailed questions than can be discussed within this article (www.phru.nhs.uk/casp/critical_appraisal_tools.htm).
The 10 main questions that the CASP asks of qualitative research are listed in Box 2.
- Is there a clear statement of the aims of the research?
Qualitative research needs to answer questions set by the researcher (there is no hypothesis). The intended aim(s) of the research should therefore be stated and the questions the research seeks to address should be identified.
- Is a qualitative methodology appropriate?
Some research questions are best addressed by qualitative enquiry and others by quantitative enquiry. Returning to the example of hypertension, a researcher who chooses to ask patients about their experiences of receiving different treatments for hypertension is clearly seeking to use a qualitative paradigm, as the patients’ thoughts and feelings are being considered, so qualitative (non-numerical) data will be collected. It would not be possible to use the quantitative paradigm and collect numerical data for this.
- Was the research design appropriate to address the aims of the research?
The research approach normally influences its design. That is, the research design tells readers how the researcher actually implemented the research approach.
- Was the recruitment strategy appropriate to the aims of the research?
Recruitment to any research project is an important consideration but qualitative researchers often want to obtain the thoughts or opinions of a specific group of people who have experienced a phenomenon. It is more important therefore for qualitative researchers to ensure that participants have experienced the phenomenon rather than randomly selecting people who may not be able to answer the questions.
- Was data collected in a way that addressed the research issue?
Within qualitative research the most common methods of data collection are interview, questionnaire or observation. These different methods are used to obtain slightly different data. For example, it is not possible to obtain patients’ thoughts and feelings of their different treatments for hypertension by observing them. Therefore the method of data collection should address the research issue/question.
- Has the relationship between researcher and participants been adequately considered?
Within qualitative research the researcher and participant often have a ‘close relationship’, especially if the researcher returns to the participant on several occasions over a period of time - which may even run into many months. Researchers must be careful not to introduce bias by accidentally reporting their interpretation of participants’ feelings. Within some qualitative approaches (phenomenology) researchers must separate out (bracket) and declare their feelings at the beginning of the research project.
- Have ethical issues been taken into consideration?
Ethical issues are important and should be considered at every step of the research process. This is not just about obtaining ‘ethical approval’ for a study but also ensuring the rights of participants are not violated. When reporting qualitative research, participants’ anonymity and confidentiality must not be breached. The Central Office for Research Ethics Committees (www.corec.org.uk) offers guidance for researchers planning to undertake any research within the NHS.
- Was the data analysis sufficiently rigorous?
Data analysis is both time consuming and rigorous within qualitative research. Although it is easy for qualitative researchers to make assumptions and want to interpret the data quickly, they really need to be ‘immersed’ in the data over a period of time as the first stage of data analysis.
- Was there a clear statement of findings?
Once the data has been analysed thoroughly, the findings should be clearly displayed. Although there are many ways to analyse qualitative data, researchers normally organise it into common groups/topics (themes). These should be examined within the research report with examples (quotes) from each theme being given.
- How valuable is the research?
The majority of research is never published even though it may have the potential to make a valuable contribution to the development of nursing knowledge. Caution should still be exercised as to whether published research is repetitive of previous research or conducted well - hence the reason for a critiquing framework - and adds to the body of nursing knowledge.
What are the critiquing frameworks for qualitative research trying to do?
The questions in frameworks for critiquing qualitative research tend to be sequential. It is vital for readers to understand the aims and questions of the research in order to answer sections of the framework. Critiquing frameworks enable readers to make a judgement regarding the soundness of the research. While it is possible to critique a piece of research without them, frameworks serve as useful aides-memoires for those who are not used to critiquing research.
What do academic journals expect?
Many of the issues affecting quantitative research reported in the second article in this series (Lee, 2006b) also apply to qualitative research, such as restrictions on word limits. This makes it important for readers to examine author guidelines for the particular journal.
It is clearly not possible to cover every aspect of a study in a journal report. Readers should therefore be careful before boldly stating that a researcher did not consider a certain aspect, since it may have been discussed at length in the original (unpublished) research report. By reading the author guidelines of professional journals readers are better able to determine what authors can include (and therefore by implication exclude), and then relate this to the critiquing framework outlined above.
The terminology used in some professional journals would require inexperienced readers to explore meanings further. For example, the term ‘constant comparative analysis’ would be familiar to qualitative researchers - particularly those who use the grounded theory approach (methodology) even though there may not be space within the journal to explain why this is unique to one particular research approach. Once data analysis has been completed, then qualitative researchers discuss their findings, although generalisability is not normally possible with qualitative research.
Nursing journals have different expectations regarding how much discussion should be included regarding the trustworthiness of qualitative research. In fact not all qualitative researchers report these ideas in the same way - some use the notion of trustworthiness, whereas others use the terms validity and reliability, which are actually related more to quantitative research.
Finally, readers should consider implications of the research for nursing and midwifery. While these implications can relate to the practice, education, research and management of nursing and midwifery, not all research reports detail the implications for all aspects of the professions. Indeed, the word allowance in many health journals will only allow for detailed discussion of a few implications.
How qualitative research assists practitioners
Careful examination of qualitative research gives practitioners a better understanding of how a group of people view or understand a particular situation. It can therefore enable individual practitioners to enhance their practice and contribute to evidence-based practice.
This series outlines the two main approaches to research (recognising that there are other more specialised approaches) and explains how to read and critique qualitative and quantitative research (see last week’s issue for part two in the series). The development of such skills should assist pre-registration students with relevant assignments. They should also help practitioners to determine if a piece of research is relevant and suitable to be implemented in their practice. The skills outlined in this series are also essential prerequisites for those intending to undertake a critical review of literature, begin their own programme of research, or undertake systematic reviews of research.
As practitioners gain a deeper understanding of critiquing a single piece of research, they should consider critiquing several research studies on a particular topic, searching for common themes. They could then write a critical review of the literature on that chosen topic.
Each week Nursing Times publishes a guided learning article with reflection points to help you with your CPD. After reading the article you should be able to:
- Understand the nature and purpose of qualitative research;
- Know the role of critiquing frameworks;
- Understand what these frameworks aim to achieve;
- Be familiar with how qualitative research can help practitioners.
Use the following points to write a reflection for your PREP portfolio:
- Outline where you work and the relevance of this article to your practice;
- Identify the last time you came across a piece of qualitative research;
- Discuss something new you have learnt about qualitative research in this article;
- Explain how this information could have informed your care of a patient;
- Outline how you intend to disseminate what you have learnt among your colleagues.