Issues in Web-based Research


 
Page 1

1) Advantages to Web-based research.
2) Disadvantages to Web-based research.
3) What kinds of studies can you do on-line?
4) Distributing your instrument.
5) Housing your lab study on-line.
6) Response rates and attrition in Web-based research.

Page 2

7) Issues of privacy, anonymity, and confidentiality.
8) Using databases on the Internet.
9) Similarity of Internet- and lab-based studies.
10) Using your Web servers to your advantage.
11) Maximizing responses to Web-based surveys.

* Issues in Web-based Research - Page 2 *

7)  Issues of privacy, anonymity, and confidentiality in Web-based research?
        Privacy and anonymity on the Internet are popular, and real, concerns.  Frequent stories of credit card number thefts and privacy invasions leave many Internet users understandably weary of Internet use.  Coupled with the perception of research materials is just more Spam, response rates to research materials is affected.  Cho and LaRose (1999) discuss Internet surveys from the perspective of privacy violation and offer recommendations to decrease this perception of violation.
        Anonymity for information gathered of the Internet is usually attainable.  Though some data collection procedures automatically collect the IP address of the computer from which the data was sent, it is usually difficult or impossible to match that address with a specific individual.  Studies conducted via email accounts are more prone to a lack of anonymity.  However, upon receipt of the data, it can be transferred to a data file coded by subject number rather than email address.  As in all cases, it is the responsibility of the investigators to keep sensitive data protected.  Generally speaking, data transferred over the Internet could potentially be intercepted, though again, with only an IP address, a respondent is reasonably protected.
 
        Further Reading:
        Cho, H., & LaRose, R. (1999). Privacy issues in Internet surveys. Social Science Computer Review, 17, 421-434.
        Frankel, J. S., & Siang, S. (1999). Ethical and legal aspects of human subjects research on the Internet. Retrieved August 3, 2003, from http://www.aaas.org/spp/dspp/sfrl/projects/intres/report.pdf.
        O'Neil, D. (2001). Analysis of Internet users' level of online privacy concerns, Social Science Computer Review, 19, 17-31.
 

8) Using databases on the Internet.
        Though the work on this website is primarily about tools for gathering information, there are good Web-based tools for analyzing previously gathered information.  For example, the General Social Survey offers access to a wide variety of social science variables that can be analyzed from the website.  Some of these sites are searchable.  These can be good tools for making students think about relationships between variables; they can develop hypotheses and then test them.

        Sample websites:

        Notes: a) If you have more suggestions, please send them along to me; b) Many thanks to Dr. Loren Demorath of Centenary College for providing information regarding these database sites.

9)  Similarity of Internet- and lab-based studies.
        Research conducted to date supports the similarity of results between web-based and lab-based studies (see cites below).  For example, analysis of data conducted over the PsychExps online lab found similarities of both within-subjects and between-subjects effects (McGraw, Tew, & Williams, 2000).  Regarding the additional experimental variance undoubtably obtained by having participants complete studies in largely uncontrolled environments, McGraw and his colleagues concluded "...the added noise...is easily compensated for by the sample saized achievable with Web delivery" (p. 502).  This does not mean sampling is not a big concern.

     * Further reading:
        Birnbaum, M. H. (1999). Testing critical properties of decision making on the Internet.  Psychological Science, 10, 399-407.
        Birnbaum, M. H. (2000). Psychological Experiments on the Internet. San Diego, CA:  Academic.
        Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum's (Ed.)  Psychological Experiments on the Internet. San Diego, CA:  Academic.
        McGraw, K. O., Tew, M. D., & Williams, J. E. (2000). The integrity of Web-delivered experiemnts:  Can you trust the data?  Psychological Science, 11, 502-506.

10) Using your Web server to your advantage
        While most of us are used to viewing webpages with browsers (e.g., Netscape Navigator, Internet Explorer), our interaction with the millions of webpages on the Web is controlled via Web servers.  Programming a Web server to be your allia in web-based researcher can offer many benefits (e.g., real-time checking of respondent information for errors, immediate feedback to responses, real-time calculations).  However, this project focuses on tools which limit the need for such expertise.  Thus, I'll offer a few places to start (see below), but leave you to researcher this issue on your own.

     * Further reading:
        Schimdt, W. C. (2000). The server side of psycholoby web experiments. In M. H. Birnbaum's (Ed.) Psychological Experiments on the Internet. San Diego, CA: Academic.
        Schmidt, W. C., Hoffman, R., & MacDonald, J. (1997). Operate your own World-Wide Web server. Behavior Reserach Methods, Instruments, & Computers, 29, 189-193.

11) Maximizing responses to a Web-based survey
        As with any survey tool, there are characteristics of the instrument themselves which influence completion rate.  For example, graphics and graphical sliding tools are nice, but the additional download time might make them too cumbersome for many respondents (Couper, Traugott, & Lamias, 2003).
        Dillman and colleagues (Dillman, Tortora, & Bowker, 1999) point out 4 potential problem areas when conducting Web-based surveys:  Coverage error, Sampling error, Measurement error, and Nonresponse error.  Failing to adequately address any of these can seriously empare the quality of your data.  They offer a number of specific principles to mazimize responses.

     * Further reading:
        Couper, M. P., Traugott, M. W., & Lamias, M. J. (2003). Web survey design and administration.  Public Opinion Quarterly, 65, 230-253.
        Truell, A. D., Bartlett, J. E., & Alexander, M. W. (2002). Response rate, speed, and completeness: A comparison of Internet-based and mail surveys. Behavior Research Methods, Instruments, & Computers, 34, 46-49.
    Dillman, Tortora, & Bowker, (1999). Principles for Constructing Web Surveys.
    Reips, U.-D. (2002). Internet-based psychological experimenting:  Five dos and five don'ts. Social Science Computer Review, 20, 241-249.



If you have questions or problems, please let me know.
Last updated: 8/15/03