Web-based experiments

A web-based experiment or Internet-based experiment is an experiment that is conducted over the Internet. In such experiments, the Internet is either "a medium through which to target larger and more diverse samples with reduced administrative and financial costs" or "a field of social science research in its own right."[1] Psychology and Internet studies are probably the disciplines that have used these experiments most widely, although a range of other disciplines including political science and economics also use web-based experiments. Within psychology most web-based experiments are conducted in the areas of cognitive psychology and social psychology.[2][3] This form of experimental setup has become increasingly popular because researchers can cheaply collect large amounts of data from a wider range of locations and people. A web-based experiment is a type of online research method.

Introduction

Experiments are an integral part of research, however, their integration with the Internet has been gradual. There are three main categories of experiments:

The adaption of each type of experiment online faces some hurdles.

Benefits

Web-based experiments are significantly less expensive, potentially allowing the researcher to:

These benefits have the potential to translate into greater external validity and generalizability for the study. For instance, in web-based experiments there is less reliance on data gathered from populations of Western undergraduate students who are often used as the default research subjects in social science disciplines.[8] Because participants remain in their homes or offices while participating in the experiment, scholars have also argued that such experiments have greater ecological validity.[4]

Criticisms and limitations

Web-based experiments may have weaker experimental controls compared to laboratory-based experiments, and may face greater difficulties coming up with procedures that ensure reliability and internal validity.[9] Online natural and field experiments may also face challenges generalizing findings beyond the online context in which they were conducted. Some potential difficulties faced by web-based experiments include:

In the face of these criticisms, some researchers have argued that brick-and-mortar experiments are just as affected by these problems, if not more so.[14][15][16]

Studies have been conducted to test the internal validity of web-based experiments, comparing across experimental conditions (online and offline) and successfully replicating findings. For example, Schoeffler et al. (2013) compared laboratory- and web-based results (62 and 1,168 subjects) of an auditory experiment and found no significant differences.[17] A paired experiment in behavioral economics split into online and traditional lab environments produced substantively similar results.[1] Uncompensated and unsupervised subjects on LabintheWild have been shown to replicate previous in-lab study results with comparable data quality.[18]

Methodologies

Experimental protocols have been suggested to prevent or control difficulties associated with web-based experimentation. Methods like sequential subject matching, background timing and mouse use tracking, and instantaneous compensation through PayPal have the potential to address many of the concerns about the internal validity of web-based experiments.[1] These methods control for differences in response times, address issues of selective attrition, concentration, and distraction, minimize subject concern about compensation, improve subject confidence that they have a real human partner in the experiment, and ensure that subjects have an appropriate understanding of the instructions and the decision problems in the experiment.[1]

Scholars have also formulated techniques to decrease or account for drop-outs, including the high-hurdle technique (motivationally adverse information is clustered at the beginning of the study), the seriousness check (requesting participant's probability estimate that they'll complete the study), and the warm-up phase (placing consent forms or other pre-study materials first to winnow the samples before the study begins).[4]

Examples

Use in psychology

A wide range of psychology experiments are conducted on the web. The Web Experiment List provides a way to recruit participants and archives past experiments (over 700 and growing).[19] A good resource for designing a web experiment is the free Wextor tool, which "dynamically creates the customized Web pages needed for the experimental procedure" and is remarkably easy to use.[20] Web experiments have been used to validate results from laboratory research and field research and to conduct new experiments that are only feasible if done online.[4] Further, the materials created for web experiments can be used in a traditional laboratory setting if later desired.

Interdisciplinary research using web experiments is rising. For example, a number of psychology and law researchers have used the web to collect data. Lora Levett and Margaret Bull Kovera examined whether opposing expert witnesses are effective in educating jurors about unreliable expert evidence.[21] Rather than sensitizing jurors to flaws in the other expert's testimony, the researchers found that jurors became more skeptical of all expert testimony. In her experiment, this led to more guilty verdicts. Levett and Kovera's research used a written transcript of a trial, which participants then read before making their decision. This type of stimulus has been criticized by some researchers as lacking ecological validity—that is, it does not closely approximate a real-life trial. Many recommend the use of video where possible. Researchers at New York University are currently conducting a psychology and law study that uses video of a criminal trial.[22]

Researchers at University of Salford are currently conducting a number of studies online to explore sound perception.[23] Sound experiments over the web are particularly difficult due to lack of control over sound reproduction equipment.

Salganik, Dodds, and Watts conducted an experiment to measure social influence, specifically in the popularity rating of songs. Their use of the Internet allowed them to collect over 14,000 participants and examine the relationship between individual and collective behavior.[6]

Use in economics

As more experiments have been conducted in economics, questions about appropriate methodology and study organization has been raised. Jerome Hergueux and Nicolas Jacquemet developed an "online laboratory" to compare social preferences and risk aversion online and in person. They administered a risk aversion assessment, Public good game, a Trust game, a Dictator game, and an Ultimatum game to groups both online and in a lab as a way of assessing the internal validity of web-based experimentation in economics.[1]

Use in political science

An online field experiment conducted on 61 million Facebook users tested whether receiving information about voting, polling places, and the voting behavior of one's friends led individuals to seek out political information, influenced political self-expression, and changed real-world voting behavior.[7]

Use in Internet studies

Web-based experiments have particular salience in studies of how online communities operate. Internet studies, including studies of online communities and social networks, have used natural and field experiments to understand the effects of informal rewards in peer production on Wikipedia,[24][25] as well as the impact of early recognition and support on future successes on Kickstarter, Change.org, Epinions, and Wikipedia.[26] Another experiment looked at the effect on edit rates of introducing a program of intelligent task assignment on Wikipedia.[27]

References

  1. 1 2 3 4 5 6 Hergueux, J., & Jacquemet, N. (2014). Social preferences in the online laboratory: a randomized experiment. Experimental Economics, 18(2), 251–283.
  2. ↑ Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford: Oxford University Press.
  3. ↑ Reips, U.-D. & Krantz, J. H. (2010). Conducting true experiments on the Web. In S. Gosling & J. Johnson, Advanced Internet Methods in the Behavioral Sciences (pp. 193-216). Washington, DC: American Psychological Association.
  4. 1 2 3 4 5 6 7 Ulf-Dietrich Reips, "Standards for Internet-Based Experimenting", 49 Experim. Psych. 243 (2002)
  5. ↑ Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790.
  6. 1 2 Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006). "Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market". Science, 311(5762), 854–856.
  7. 1 2 Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415), 295–298.
  8. ↑ Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83.
  9. ↑ Hoffman, M., & Morgan, J. (2011). Who’s Naughty? Who’s Nice? Social Preferences in Online Industries. UC Berkeley Working Paper.
  10. ↑ Anderhub, V.; Müller, R.; Schmidt, C. (2001). "Design and evaluation of an economic experiment via the Internet". Journal of Economic Behavior & Organization.
  11. ↑ Kahneman, D. (2003). "Maps of bounded rationality: Psychology for behavioral economics". The American Economic Review.
  12. ↑ Shavit, T.; Sonsino, D.; Benzion, U. (2001). "A comparative study of lotteries-evaluation in class and on the Web". Journal of Economic Psychology.
  13. ↑ Eckel, C. C.; Wilson, R. K. (2006). "Internet cautions: Experimental games with Internet partners.". Experimental Economics.
  14. ↑ Reips,U.-D. (1996, October). Experimenting on the World WideWeb. Paper presented at the 1996 Society for Computers in Psychology conference, Chicago.
  15. ↑ "Virtual labs: Is there wisdom in the crowd?". New Scientist. 16 March 2007.
  16. ↑ http://www.apa.org/monitor/apr00/research.html
  17. ↑ Schoeffler et al. (2013). "An Experiment About Estimating the Number of Instruments in Polyphonic Music: A Comparison Between Internet and Laboratory Results"
  18. ↑ Reinecke, Katharina; Gajos, Krzysztof Z. (2015-01-01). "LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples". Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. CSCW '15. New York, NY, USA: ACM: 1364–1378. doi:10.1145/2675133.2675246. ISBN 9781450329224.
  19. ↑ http://wexlist.net
  20. ↑ http://wextor.org
  21. ↑ Levett & Kovera, "The Effectiveness of Opposing Expert Witnesses for Educating Jurors about Unreliable Expert Evidence", 32 L. & Hum. Behav. 363 (2008)
  22. ↑ Virginia vs. McNamara
  23. ↑ http://www.sound101.org/
  24. ↑ Restivo, Michael; Rijt, Arnout van de (2012-03-29). "Experimental Study of Informal Rewards in Peer Production". PLOS ONE. 7 (3): e34358. doi:10.1371/journal.pone.0034358. ISSN 1932-6203. PMC 3315525Freely accessible. PMID 22479610.
  25. ↑ Zhu, Haiyi; Zhang, Amy; He, Jiping; Kraut, Robert E.; Kittur, Aniket (2013-01-01). "Effects of Peer Feedback on Contribution: A Field Experiment in Wikipedia". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '13. New York, NY, USA: ACM: 2253–2262. doi:10.1145/2470654.2481311. ISBN 9781450318990.
  26. ↑ Rijt, Arnout van de; Kang, Soong Moon; Restivo, Michael; Patil, Akshay (2014-05-13). "Field experiments of success-breeds-success dynamics". Proceedings of the National Academy of Sciences. 111 (19): 6934–6939. doi:10.1073/pnas.1316836111. ISSN 0027-8424. PMC 4024896Freely accessible. PMID 24778230.
  27. ↑ Cosley, Dan; Frankowski, Dan; Terveen, Loren; Riedl, John (2007-01-01). "SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia". Proceedings of the 12th International Conference on Intelligent User Interfaces. IUI '07. New York, NY, USA: ACM: 32–41. doi:10.1145/1216295.1216309. ISBN 1595934812.

External links

This article is issued from Wikipedia - version of the 11/12/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.