This article explores the concept of using online questionnaires to carry out nursing research. It discusses options for nurses who do not have advanced technical IT skills for electronic distribution of survey questionnaires.
The general principles of web distribution are explained, and some approaches are evaluated in terms of current access to technology and its ease of use. The article also offers some practical advice for nurse researchers, and examines the advantages and disadvantages of using this new method of data collection, compared with traditional hard copy.
Jones, S. et al (2008) Using online questionnaires to conduct nursing research. Nursing Times; 104: 47, 66–69.
Steve Jones, MBA, BA, CertEd (FE), RMN, DN, is lecturer and head of information technology; Fiona Murphy, MSc, PGCE (FE), BN, NDN, RCNT, HV, RN, is senior lecturer; Mark Edwards, BSc, RN, is tutor; Jane James, MSc, RN, is tutor; all at School of Health Science, University of Wales Swansea.
Increasingly, there is an expectation that nurses and other health care professionals will be involved in research and audit activities and will use the findings from these activities to inform their clinical practice. They are expected to be able to critically appraise evidence from research, be involved in conducting and contributing to audit and be actively involved in undertaking research projects themselves. This requires knowledge about research and in particular the methods that could be used to collect data.
A popular method of collecting data is through surveys, which involves using questionnaires. These are designed through the careful construction of questions to identify facts and opinions from specific groups of respondents (Denscombe,
2003). Using a questionnaire to gather research data is often an attractive proposition as they are arguably more precise and focused than alternative methods such as interviewing and observation by researchers.
The advantages and disadvantages of their use have been widely debated (McKenna et al, 2006; Giuffre, 1997). The main advantage of well-constructed questionnaires appears to be that they enable researchers to easily collect relatively unambiguous data which lends itself to quantitative data analysis (Bowling, 2002). However, questionnaires also have their disadvantages, one of which is that they often achieve a poor rate of return, leading to low response rates.
Two further problems relate to issues of control and transcription. The control issue occurs where researchers have no control over the order in which respondents answer questions (which might be critical to the study), no facility to check on incomplete responses or incomplete questionnaires, and no ability to prevent the questionnaires being passed on to others (Oppenheim, 1992). The transcription issue relates to the task of accurately reading the data from the finished questionnaire – and manually entering it into the analysis software. This is where transcription errors may creep in – particularly where a respondent’s handwriting is difficult to decipher.
Implications for practice
Questionnaires can be administered in a variety of ways. They can be distributed by hand or by post, or sent by email to be printed out, completed and then returned.
A newer alternative approach is to deliver the questionnaire entirely electronically via the internet, as an online questionnaire. Respondents access the web page, read the questionnaire and enter their responses directly on to the page.
Online questionnaires have the advantages of hard copy but also have features which make spoiled and incomplete responses impossible to submit. For example, respondents can be guided through the process to ensure they complete the questionnaire fully, properly and in the correct order before they are able to submit it (Solomon, 2001). This overcomes the control issue associated with hard copy.
Another advantageous feature of online questionnaires is that the data analysis tools will either be an integral part of the website or data can be copied or ported directly into analysis software such as SPSS or Microsoft Excel. Typically both options are offered. This means the results are available as the data is entered, and transcription errors and the chore of manual data entry into separate analysis software are eliminated.
However, the problem of low response rates associated with questionnaires remains. In fact there is evidence that the rates are lower for online surveys than for postal surveys (Solomon, 2001). Although these findings are from research carried out almost a decade ago they may still apply.
In a large study conducted in Norway, researchers found that adding the option to respond to a questionnaire via the internet did not significantly increase the overall response to a postal survey (Brøgger et al, 2007). In another large study in Sweden,
the return rate for the online version of a questionnaire was 15% compared with 55% for the postal version (Dannetun et al, 2007).
However, it is worth noting that in both the above studies participants were selected randomly. Whitehead (2007) found evidence that response rates for internet-mediated surveys are lower when the request is unsolicited. However, they can still be as high or higher than postal surveys in which people are recruited either face-to-face or online.
Research in practice
An assumption behind this article is that nurses undertaking research are not likely to be targeting large numbers of people at random. Since email is increasingly used as the system of choice for keeping in touch with patient groups, it seems reasonable that web links to research questionnaires could be appended or attached to emailed documents such as appointment reminders, in the same way they are sometimes enclosed with postal versions.
In the surveys cited where low response rates have been reported (Brøgger et al, 2007; Dannetun et al, 2007), login credentials have always been sent to potential respondents by post. The Swedish respondents were offered a choice of completing the form contained in the envelope they had just opened, or turning on their computer, finding the site, successfully logging in and then completing and submitting the same questionnaire online (Dannetun et al, 2007).
The proposal here is that links to questionnaires should be integrated into patients’ ordinary interaction with the health service. They will be sitting at their computer and logged in when they first encounter the questionnaire. So, while it seems people are reluctant to go to the trouble of responding to a web address printed on hard copy – particularly if this is unsolicited – the invitation to click on a link in an already open email is probably a more attractive option. It is also less intrusive than either hard copy or telephone follow-up and works asynchronously, in that patients can respond in their own time.
Including a questionnaire within routine support from their usual health care provider also offers patients assurance they are not responding to a commercial study, as may happen if they arrive at a questionnaire through using a search engine (Etter, 2006). A wide range of data could be collected using questionnaires in this way, such as feedback on the nature and type of post-operative pain or discomfort patients experience, or on particular difficulties they have encountered after discharge. They could even be used to give patients the opportunity to rate aspects of care they received while in hospital.
The question remains as to whether or not the target client group for web questionnaires exists, even assuming that nurses are now beginning to embrace a technology they have been hitherto slow to use and adopt (Im and Chee, 2001). However, there is no doubt that in the general population people are more technologically competent than ever before.
Whitehead (2007) identified older adults to be the fastest growing group of internet users. Since this is the most likely target demographic for ex-patient surveys run by nurses it seems likely there is a growing body of potential respondents, as well as access to the sort of technology needed to deliver a survey electronically.
Delivering online questionnaires
To successfully administer an online questionnaire it is necessary to have some degree of IT knowledge or at least to have access to someone with that knowledge. It is probably appropriate to explain here how an online survey works.
Most everyday computer systems still work on the ‘client server’ principle. The ordinary person uses a client computer, which interacts with a remote server computer holding their information store. Data is sent between the client computer, where it is processed, and the server where it is stored.
At the client end is the web-browser, such as Internet Explorer, or perhaps Fire Fox, which loads and displays the questionnaire pages from the website located on the remote server. Any data entered into a questionnaire at the client end is transferred to the website on the remote server, usually by hitting a submit button.
Provided a researcher has the technical skills, or access to someone else who does, the easiest way to manage the delivery of an online questionnaire and the subsequent collection, analysis and storage of data is to have complete control of both ends of this operation.
Unfortunately, this approach is really only suitable for technically able people or for those with unrestricted access to sophisticated IT support. Even today, releasing an online survey is not as simple as creating a questionnaire in Microsoft Word, copying it to a website and then sending around the web address. Any data entered into the questionnaire has
to be passed to a database somewhere.
To create this database and link it to the questionnaire is quite technically demanding, and the systems that permit
this such as Adobe Enterprise Server are not suitable for people who have only average IT skills.
However, this is far from the only way to deliver an online questionnaire. There are at least three simpler alternatives (Box 1), all three of which involve renting or borrowing space on someone else’s computer, putting the questionnaire on that, and letting the service provider, who has all the IT expertise required, manage the collection and storage of the data.
Box 1. Delivery of online questionnaires
A number of methods can be used to create and deliver online questionnaires:
As usual, in doing things this way, there is a trade-off between the amount of flexibility in the questionnaire creation tools and their ease of use. And, although typically time-limited or restricted demonstrations are free, there will always be a charge for the full version of these products and services.
Using an existing service
The first, simplest and cheapest approach of all, but one really only open to educators, is to exploit the capabilities of whatever virtual learning environment (VLE) is used by the individual researcher’s organisation. The two market leaders are Blackboard – which is used in many universities, and Moodle – the choice of the Open University and many NHS trusts. Both have questionnaire creation modules. A disadvantage to most researchers, other than the simplicity of what is possible, is that potential respondents would need to have an account on the VLE; this means they need to be students at least in name.
Adding a PC application
A better option might be Email Questionnaire 4.15 by CompressWeb, which costs just over £50 for the standard version. This is an example of an application that runs on the PC within Microsoft Outlook or Outlook Express and allows the construction of quite sophisticated questionnaires. It also provides an extensive library of ready-made templates and a variety of reports.
A disadvantage for novices is that they may need to know something about their email service, the name, for example, of their ‘PoP server’ – the computer which sends them their internet email. Another requirement is that the software be installed on the PC rather than accessed via the web. This means users need sufficient IT skills and sufficient rights on the PC to do this.
Using a web service
A simpler but slightly more expensive option is to use a web-based questionnaire builder. This requires little more IT skill than the ability to create an account on a website. The site will not interact with the user’s PC other than to send emails confirming, for example, that ‘your questionnaire is launched’. The questionnaire can be distributed by email attachment, by simply sending the web link to potential respondents, but it is not restricted to email recipients running particular email programmes as
it runs entirely within the web-browser.
An example, currently costing around £175, is ‘Zoomerang’. Like Email Questionnaire 4.15, Zoomerang offers a range of questionnaire templates. An advantage of using a template, rather than designing a screen from scratch, is that there is no need to worry about design or technical issues, such as keeping the relationship between the elements on the page consistent at different screen resolutions. All this is taken care of, and logos and graphics can be easily added to the basic pages.
In this article only one application of each type has been examined. Table 1 offers a list of similar alternatives, which would work in similar ways (this list is not exhaustive). Wright (2005) offered a comparative analysis of such products (company mergers have taken place since that evaluation). Of the applications evaluated, only one seems to allow questions to be spread over a number of pages but this may be no bad thing as the requirement to scroll may increase the speed at which questionnaires are completed (Manfreda et al, 2002), and simplify navigation.
Table 1. List of survey-hosting companies on the internet
Electronic data poses a special challenge. Unlike hard copy, it is easy to transmit and duplicate and will be collected using the internet, the electronic infrastructure that supports the worldwide web. Before using electronic means of data collection, in addition to providing the usual letter of explanation and asking for patients’ consent, nurse researchers should ensure the system chosen is acceptable to local IT management, and is in line with local and national connection policies. In practice, it would be better to approach the IT department early and ensure they are happy about the plans before arranging the normal ethical review, as this might save some work. It is also possible they will have encountered people in the area using web questionnaires, or will know of reliable service providers, and will be able to provide some specific support.
Another consideration is that delivering a questionnaire by email, or via patients’ login to a service, targets them personally. This may convince them, despite reassurances to the contrary, that they can be identified in a completed return. Sometimes this may not matter, as they may want to be identified. However, where anonymity is an issue, fear of breach of confidentiality has been identified as a key element affecting survey response rates (Dillman, 2000) and fears of identification associated with non-response (Saewyc et al, 2004; Morrel-Samuels, 2003). Recent research has identified this as a significant determinant of the quality and level of return, even in a highly structured environment with full control over the technology, with participants invited to attend timetabled sessions in PC labs to respond to the questionnaire (Jones et al, 2008).
It is also important to consider the security of data collected. If an individual’s record could be identified from the data collected, then the obligations towards personal data under the Data Protection Act 1998 will apply. If, on the other hand, the data is fully anonymised it will not apply. Difficulties exist where, although personal identifiers are not collected, it may still be possible to deduce patients’ identity from other data in the record. If there is any doubt, clearance should be sought from the organisation’s data protection officer. The Parliamentary Office of Science and Technology has provided advice on this point. In addition, data collected and downloaded will need to be stored safely.
Questionnaires will remain an important method of data collection. The opportunity to deliver these via the internet brings some key advantages but also some challenges. The main issue currently is the need to have a degree of IT expertise in order to distribute the questionnaires. The technology remains a challenge, both for would-be researchers and for respondents. It seems responses tend to come mainly from experienced computer users who find completing a questionnaire with a mouse easy and straightforward (Kiernan et al, 2005). People who are not as accustomed to using computers need some preparation, support and follow-up (Hayslett and Wildemuth, 2004).
Rising to this challenge will become easier as internet use increases and new products arrive on the market designed to allow professionals with no specialist IT skills to create their own online questionnaires. Although there is always a learning curve
in using the internet to conduct surveys, like most applications of technology, once basic skills have been learnt the relative effortlessness of this method makes the earlier ‘hard copy’ approach unappealing. Other important considerations, such as assuring respondents of the anonymity and confidentiality of their participation, remain a challenge.
Bowling, A. (2002) Research Methods in Health. Investigating Health and Health Services (2nd ed). Buckingham: Open University Press.
Brøgger, J. et al (2007) No increase in response rate by adding a web response option to a postal population survey: a randomized trial. Journal of Medical Internet Research; 9: 5, e40.
Dannetun, E. et al (2007) Parents’ attitudes towards hepatitis B vaccination for their children. A survey comparing paper and web questionnaires, Sweden 2005. BMC Public Health; 7: 86.
Denscombe, M. (2003) The Good Research Guide for Small Scale Research Projects (2nd ed). Maidenhead: Open University Press.
Dillman, D.A. (2000) Mail and Internet Surveys: The Tailored Design Method. New York, NY: Wiley.
Etter, J. (2006) Internet-based smoking cessation programs. International Journal of Medical Information; 75: 1, 110–116.
Giuffre, M. (1997) Designing research survey design: part two. Journal of PeriAnesthesia Nursing; 12: 5, 358–362.
Hayslett, M.M., Wildemuth, B.M. (2004) Pixels or pencils? The relative effectiveness of web-based versus paper surveys. Library and Information Science Research; 26: 1, 73–93.
Im, E., Chee, W. (2001) A feminist critique of the use of the internet in nursing research. Advances in Nursing Science; 23: 4, 67–82.
Jones, S. et al (2008) Doing things differently: advantages and disadvantages of web questionnaires. Nurse Researcher; 15: 3, 15–26.
Kiernan, N.E. et al (2005) Is a web survey as effective as a mail survey? A field experiment among computer users. American Journal of Evaluation; 26: 2, 245–252.
Manfreda, K.L. et al (2002) Design of web survey questionnaires: three basic experiments. Journal of Computer Mediated Communication; 7: 3.
McKenna, H.P. et al (2006) Surveys. In: Gerrish, K., Lacey, A. (eds) The Research Process in Nursing (5th ed). Oxford: Blackwell Publishing.
Morrel-Samuels, P. (2003) Web surveys hidden hazards. Harvard Business Review; 81: 7, 16–18.
Oppenheim, A.N. (1992) Questionnaire Design, Interviewing and Attitude Measurement. London: Pinter Publishers.
Saewyc, E.M. et al (2004) Measuring sexual orientation in adolescent health surveys: evaluation of eight school-based surveys. Journal of Adolescent Health; 35: 4, 345e1–15.
Solomon, D.J. (2001) Conducting web-based surveys. Practical Assessment, Research and Evaluation; 7: 19.
Whitehead, L.C. (2007) Methodological and ethical issues in internet-mediated research in the field of health: an integrated review of the literature. Social Science and Medicine; 65: 4, 782–791.
Wright, K.B. (2005) Researching internet-based populations: advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication; 10: 3, article 11.