Abstract
Many studies examining on the comparison of computer-based tests and paper-and-pencil tests are available in the literature. These studies are applied to various fields, variables and workgroups. So it is important to compile and review these studies by using meta-analysis for reaerchers and authorities. In this paper, a survey of the studies on comparison of student performances in paper-based and computer-based tests is presented. By considering these studies, a meta-analysis on the comparison of students’ achievements for paperbased and computer-based tests is provided. To determine if the studies included into meta-analysis have bias, Funnel Graph and Begg-Mazumdar correlation test are utilized for sensitivity analysis. Influence quantities pertaining the studies are analyzed by using Comprehensive Meta-Analysis software. Results of heterogeneity test are analyzed to determine the best effect model and then it is seen that effect quatities shows normal distribution. Based on that, Fixed Effect Model is selected. Hedge’s g coefficient is used to calculate the common effect. There are two basic restrictions of this study. First, due to the lack of time, it could not be possible to reach all studies available for the comparison of paper-based and computer-based tests. Second, results of the meta-analysis are not compared with respect to the topic fields. It could be better to compare the results for topic fields as future prospect. According to 35 findings related to the 9 different studies included to the meta-analysis, no reasonable difference is observed for paper-based and computer-based tests for 29 comparisons, and some reasonable differences are observed 6 ones. Common influence quantity obtained from all the studies is found as -0.09. This necessarily means that the differences between scores obtained from paper-based and computer-based tests corresponds to 0.09 standard deviation, which is not a considerable influence quantity. In other words, according to the meta-analysis results, there is no substantial difference for the students’ performance in paper-based and computer-based tests. As a result, even though it is not inconvenient to use paper-based tests, it should be determined while preparing a computer-based test whether student achievement is affected by applied test form via some trial applications.
- Aybek, E. C. (2012). Kağıt-Kalem Formu ve Bilgisayar Ortamında Uygulanan Genel Yetenek Testinin Psikometrik Özelliklerinin Karşılaştırılması, Yayınlanmamış Yüksek Lisans Tezi, Ankara Üniversitesi Eğitim Bilimleri Enstitüsü.
- Bayazıt, A. (2007). Çevrimiçi Sınavlar Ve Kâğıt-Kalem Sınavları Arasındaki Sınav Süresi ve Öğrenci Başarım Farklılıkları, Yayınlanmamış Yüksek Lisans Tezi, Hacettepe Üniversitesi Fen Bilimleri Enstitüsü.
- Boo, J., & Vıspoel, W. (2012). Computer Versus Paper-And-Pencıl Assessment Of Educatıonal Development: A Comparıson Of Psychometrıc Features And Examınee Preferences 1. Psychological Reports, 111(2), 443–460. doi:10.2466/10.03.11.PR0.111.5.443-460
- Bugbee Jr., A. C., Bernt, F. M. (1990). Testing by computer: Findings in six years of use 1982-1988. Journal of Research on Computing in Education, Vol. 23 Issue 1, 87-101.
- Chan-Pensley, E. (1999). Alcohol-Use Disorders Identification Test: a comparison between paper and pencil and computerized versions. Alcohol and alcoholism (Oxford, Oxfordshire), 34(6), 882–885. 04.04.2013 tarihinde http://www.ncbi.nlm.nih.gov/pubmed/10659724 adresinden alınmıştır.
- Drasgow, F (2002). The work ahead: A psychometric infrastructure for computerized adaptive tests. In C.N. Mills,
- M.T. Potenza, J.J. Fremer, & W.C. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 67–88). Hillsdale, NJ: Lawrence Erlbaum.
- Durlak, J.A. (1995). Understanding meta-analysis. In L.G. Grimm, & P.R. Yarnold (Eds.), Reading and understanding multivariate statistics (pp. 319-352). Washington, DC: American Psychological Association.
- Helgeson, S. L., Kumar, D. D. (1993). A Review of Educational Technology in Science Assessment. National Center for Science Teaching and Learning, Columbus, Monograph Series Number 7, OH., (ERIC Document Reproduction Service No. ED366507).
- Kim, D., & Huynh, H. (2007). Comparability of computer and paper-and-pencil versions of algebra and biology assessments. The Journal of Technology, Learning and Assessment, 6(4), 1–29. 08.05.2013 tarihinde http://napoleon.bc.edu/ojs/index.php/jtla/article/view/1634 adresinden alınmıştır.
- Kumar, D. (1996). Computers and Assessment in Science Education. ERIC Clearinghouse for Science, Mathematics and Environmental Education, Columbus, OH., (ERIC Document Reproduction Service No. ED395770).
- Luecht, R. M. (2001). Challenges of Web-Based Assessment. (ERIC Document Reproduction Service No. ED458294).
- Mark Pomplun, Sharon Frey and Douglas F. Becker (2002). The Score Equıvalence Of Paper-And-Pencıl And Computerızed Versıons Of A Speeded Test Of Readıng Comprehensıon. Educational and Psychological Measurement 2002 62: 337.
- Natal, D. (1998). On-Line Assessment: What, Why, How. Imagen Multimedia Corp., Lompoc, CA., (ERIC Document Reproduction Service No. ED419552).
- Özalp-Yaman, Ş., & Çağıltay, N. E. (2010). Paper-based versus computer-based testing in engineering education. IEEE EDUCON Education Engineering (pp. 1631–1637). 05.05.2013 tarihinde http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5492397 adresinden alınmıştır.
- Poggio, J., & Glasnapp, D. (2005). A comparative evaluation of score results from computerized and paper & pencil mathematics testing in a large scale state assessment program. The Journal of Technology, Learning and Assessment, 3(6), 1–30. 08.05.2013 tarihinde http://escholarship.bc.edu/ojs/index.php/jtla/article/view/1659 adresinden alınmıştır.
- Pomplun, M., Frey, S., & Becker, D. F. (2002). The Score Equivalence of Paper-and-Pencil and Computerized Versions of a Speeded Test of Reading Comprehension. Educational and Psychological Measurement, 62(2), 337–354. doi:10.1177/0013164402062002009
- Sampson, J.P. (2000). Using the İnternet to enhance testing in counseling. Journal of Counseling and Development
- Vispoel, W. P., Boo, J., & Bleiler, T. (2001). Computerized and Paper-and-Pencil Versions of the Rosenberg Self-Esteem Scale: A Comparison of Psychometric Features and Respondent Preferences. Educational and Psychological Measurement, 61(3), 461–474. doi:10.1177/00131640121971329
- Vrabel M. Computerized versus paper-and-pencil testing methods for a nursing certification examination: a review of the literature. Comput Inform Nurs 2004;22:94-8.
AMA 10th edition
In-text citation: (1), (2), (3), etc.
Reference: Aybek E, Şahin D, Eriş H, Şimşek A, Köse M. META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST. Asian Journal of Instruction. 2014;2(2), 18-26.
APA 6th edition
In-text citation: (Aybek et al., 2014)
Reference: Aybek, E., Şahin, D., Eriş, H., Şimşek, A., & Köse, M. (2014). META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST. Asian Journal of Instruction, 2(2), 18-26.
Chicago
In-text citation: (Aybek et al., 2014)
Reference: Aybek, Eren, Döndü Şahin, Hayriye Eriş, Ahmet Şimşek, and Mustafa Köse. "META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST". Asian Journal of Instruction 2014 2 no. 2 (2014): 18-26.
Harvard
In-text citation: (Aybek et al., 2014)
Reference: Aybek, E., Şahin, D., Eriş, H., Şimşek, A., and Köse, M. (2014). META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST. Asian Journal of Instruction, 2(2), pp. 18-26.
MLA
In-text citation: (Aybek et al., 2014)
Reference: Aybek, Eren et al. "META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST". Asian Journal of Instruction, vol. 2, no. 2, 2014, pp. 18-26.
Vancouver
In-text citation: (1), (2), (3), etc.
Reference: Aybek E, Şahin D, Eriş H, Şimşek A, Köse M. META-ANALYSIS OF COMPARATIVE STUDIES OF STUDENT ACHIEVEMENT ON PAPER-PENCIL AND COMPUTER-BASED TEST. Asian Journal of Instruction. 2014;2(2):18-26.