Does the Digital Tools-supported Teaching Have an Impact on Students’ Learning Results:Meta-Analysis Evidence from 137 Experiments and Quasi Experiments

  • Hao Lei ,
  • Xue Li
Expand
  • Institute of Curriculum and Instruction, East China Normal University, Shanghai 200062, China

Accepted date: 2022-09-09

  Online published: 2022-10-27

Abstract

In order to clarify the impact of digital tools-supported teaching on students’ learning outcomes, this study conducted a meta-analysis based on 137 experiments and quasi-experiments. The results show that compared with traditional teaching, digital tools-supported teaching can enhance learning motivation (g=0.482) and improve academic achievement (g=0.479). In addition, this positive effect was moderated by demographics, knowledge assessments, and study attributes.The specific manifestations were as follows: in terms of demographic, the use of digital tools to support teaching in the context of collectivism culture had a more significant positive impact on learning motivation and academic achievement compared with individualism culture; the predictive effect of digital tools-supported teaching on learning motivation and academic achievement is not affected by the stage; the influence of digital tools-supported teaching on boys’ academic achievement is significantly higher than that of girls. In terms of knowledge assessment, compared with digital tutorials and simulation, intelligent tutoring system and hypermedia are more conducive to improving academic achievement; as opposed to other subjects, the teaching supported by digital tools has a greater impact on language learning motivation, but the teaching supported by digital tools has a significant impact on mathematics and science achievement. In terms of study attributes, the use of digital tools by trained teachers had a better impact on student motivation than did untrained teachers; only when the intervention time of digital teaching is moderate can it give full play to its best effect on academic achievement; the impact of digital tools-supported teaching on academic achievement is strengthened as the intensity of intervention increases; with the change of years, the predictive effect of digital tools- supported teaching on learning motivation will become more and more prominent.

Cite this article

Hao Lei , Xue Li . Does the Digital Tools-supported Teaching Have an Impact on Students’ Learning Results:Meta-Analysis Evidence from 137 Experiments and Quasi Experiments[J]. Journal of East China Normal University(Educational Sciences), 2022 , 40(11) : 92 -109 . DOI: 10.16382/j.cnki.1000-5560.2022.11.008

References

null Abouserie, R., Moss, D., & Barasi, S Cognitive style, gender, attitude toward computer‐assisted learning and academic achievement Educational studies 1992 18 2 151 160 Abouserie, R., Moss, D., & Barasi, S. (1992). Cognitive style, gender, attitude toward computer‐assisted learning and academic achievement. Educational studies, 18(2), 151—160.
null Aparicio, M., Bacao, F., & Oliveira, T Cultural impacts on e-learning systems' success The Internet and Higher Education 2016 31 58 70 Aparicio, M., Bacao, F., & Oliveira, T. (2016). Cultural impacts on e-learning systems' success. The Internet and Higher Education, 31, 58—70.
null Assink, M., & Wibbelink, C. J Fitting three-level meta-analytic models in R: A step-by-step tutorial The Quantitative Methods for Psychology 2016 12 3 154 174 Assink, M., & Wibbelink, C. J. (2016). Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology, 12(3), 154—174.
null Ausburn, L. J., Martens, J., Washington, A., Steele, D., & Washburn, E A cross-case analysis of gender issues in desktop virtual reality learning environments Journal of STEM Teacher Education 2009 46 3 6 Ausburn, L. J., Martens, J., Washington, A., Steele, D., & Washburn, E. (2009). A cross-case analysis of gender issues in desktop virtual reality learning environments. Journal of STEM Teacher Education, 46(3), 6.
null Bayraktar, S A meta-analysis of the effectiveness of computer-assisted instruction in science education Journal of research on technology in education 2001 34 2 173 188 Bayraktar, S. (2001). A meta-analysis of the effectiveness of computer-assisted instruction in science education. Journal of research on technology in education, 34(2), 173—188.
null Buckley, K. E., & Anderson, C. A. (2006). A theoretical model of the effects and consequences of playing video games. Playing video games: Motives, responses, and consequences, 363?378.
null Chauhan, S A meta-analysis of the impact of technology on learning effectiveness of elementary students Computers & Education 2017 105 14 30 Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14—30.
null Chen, C. M Personalized E-learning system with self-regulated learning assisted mechanisms for promoting learning performance Expert Systems with Applications 2009 36 5 8816 8829 Chen, C. M. (2009). Personalized E-learning system with self-regulated learning assisted mechanisms for promoting learning performance. Expert Systems with Applications, 36(5), 8816—8829.
null Cheung, A. C., & Slavin, R. E Effects of educational technology applications on reading outcomes for struggling readers: A best‐evidence synthesis Reading Research Quarterly 2013a 48 3 277 299 Cheung, A. C., & Slavin, R. E. (2013a). Effects of educational technology applications on reading outcomes for struggling readers: A best‐evidence synthesis. Reading Research Quarterly, 48(3), 277—299.
null Cheung, A. C., & Slavin, R. E The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis Educational research review 2013b 9 88 113 Cheung, A. C., & Slavin, R. E. (2013b). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational research review, 9, 88—113.
null Chou, C. C., Block, L., & Jesness, R A case study of mobile learning pilot project in K-12 schools Journal of Educational Technology Development and Exchange (JETDE) 2012 5 2 3 Chou, C. C., Block, L., & Jesness, R. (2012). A case study of mobile learning pilot project in K-12 schools. Journal of Educational Technology Development and Exchange (JETDE), 5(2), 3.
null Clariana, R. B., & Schultz, C. W Gender by Content Achievement in Computer-Based Instruction Journal of Computers in Mathematics and Science Teaching 1993 12 277 288 Clariana, R. B., & Schultz, C. W. (1993). Gender by Content Achievement in Computer-Based Instruction. Journal of Computers in Mathematics and Science Teaching, 12, 277—288.
null Colley, A., & Comber, C Age and gender differences in computer use and attitudes among secondary school students: what has changed? Educational Research 2003 45 2 155 165 Colley, A., & Comber, C. (2003). Age and gender differences in computer use and attitudes among secondary school students: what has changed?. Educational Research, 45(2), 155—165.
null De Jong, T., & Van Joolingen, W. R Scientific discovery learning with computer simulations of conceptual domains Review of educational research 1998 68 2 179 201 De Jong, T., & Van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of educational research, 68(2), 179—201.
null Derfler, K. E. (2002). Factors which affect middle school teachers' willingness to utilize technology as an instructional tool. La Sierra University.
null Duke, B., Harper, G., & Johnston, M. (2013). Connectivism as a digital age learning theory. The International HETL Review, 2013(Special Issue), 4?13.
null Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R Publication bias in clinical research The Lancet 1991 337 8746 867 872 Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867—872.
null Egbert, J., & Thomas, M The new frontier: A case study in applying instructional design for distance teacher education Journal of Technology and teacher education 2001 9 3 391 405 Egbert, J., & Thomas, M. (2001). The new frontier: A case study in applying instructional design for distance teacher education. Journal of Technology and teacher education, 9(3), 391—405.
null Egger, M., Smith, G. D., Schneider, M., & Minder, C Bias in meta-analysis detected by a simple, graphical test Bmj 1997 315 7109 629 634 Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. Bmj, 315(7109), 629—634.
null Ertmer, P. A., Paul, A., Molly, L., Eva, R., & Denise, W Examining teachers’ beliefs about the role of technology in the elementary classroom Journal of research on Computing in Education 1999 32 1 54 72 Ertmer, P. A., Paul, A., Molly, L., Eva, R., & Denise, W. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of research on Computing in Education, 32(1), 54—72.
null Foster II, J. J. (2001). Why Technology is Being Integrated into the Classroom at Such a Slow Rate: A Discussion of Self-Efficacy, Motivation, and Utility. In Society for Information Technology & Teacher Education International Conference (pp. 2718−2719). Association for the Advancement of Computing in Education (AACE).
null Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014).?Preparing for life in a digital age: The IEA International Computer and Information Literacy Study international report?. Springer Nature.
null Gegenfurtner, A., Quesada‐Pallarès, C., & Knogler, M Digital simulation‐based training: A meta‐analysis British Journal of Educational Technology 2014 45 6 1097 1114 Gegenfurtner, A., Quesada‐Pallarès, C., & Knogler, M. (2014). Digital simulation‐based training: A meta‐analysis. British Journal of Educational Technology, 45(6), 1097—1114.
null Guild, P The culture/learning style connection Educational leadership 1994 51 16 16 Guild, P. (1994). The culture/learning style connection. Educational leadership, 51, 16—16.
null Heidig, S., & Clarebout, G Do pedagogical agents make a difference to student motivation and learning? Educational Research Review 2011 6 1 27 54 Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning?. Educational Research Review, 6(1), 27—54.
null Heine, S. J Cultural psychology John Wiley & Sons 2010 Inc Heine, S. J. (2010). Cultural psychology. John Wiley & Sons, Inc.
null Hidi, S., Renninger, K. A., & Krapp, A. (2004). Interest, a motivational variable that combines affective and cognitive functioning. (pp. 89−115)
null Higgins, J. P., & Thompson, S. G Quantifying heterogeneity in a meta‐analysis Statistics in medicine 2002 21 11 1539 1558 Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in medicine, 21(11), 1539—1558.
null Higgins, K., Huscroft-D’Angelo, J., & Crawford, L Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis Journal of Educational Computing Research 2019 57 2 283 319 Higgins, K., Huscroft-D’Angelo, J., & Crawford, L. (2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research, 57(2), 283—319.
null Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis Computers & Education 2020 153 103897 Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153, 103897.
null Hofstede, G Cultural dimensions in management and planning Asia Pacific journal of management 1984 1 2 81 99 Hofstede, G. (1984). Cultural dimensions in management and planning. Asia Pacific journal of management, 1(2), 81—99.
null Huang, Y. M., Liang, T. H., & Chiu, C. H Gender differences in the reading of e-books: Investigating children's attitudes, reading behaviors and outcomes Journal of Educational Technology & Society 2013 16 4 97 110 Huang, Y. M., Liang, T. H., & Chiu, C. H. (2013). Gender differences in the reading of e-books: Investigating children's attitudes, reading behaviors and outcomes. Journal of Educational Technology & Society, 16(4), 97—110.
null Huedo-Medina, T. B., Sánchez-Meca, J., Marin-Martinez, F., & Botella, J Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological methods 2006 11 2 193 Huedo-Medina, T. B., Sánchez-Meca, J., Marin-Martinez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index?. Psychological methods, 11(2), 193.
null Kang, I., Choi, J. I., & Chang, K. Constructivist research in educational technology: A retrospective view and future prospects Asia Pacific Education Review 2007 8 3 397 412 Kang, I., Choi, J. I., & Chang, K. (2007). Constructivist research in educational technology: A retrospective view and future prospects. Asia Pacific Education Review, 8(3), 397—412.
null Kim, C., Park, S. W., & Cozart, J Affective and motivational factors of online math learning British Journal of Educational Technology 2014 45 1 171 185 Kim, C., Park, S. W., & Cozart, J. (2014). Affective and motivational factors of online math learning. British Journal of Educational Technology, 45(1), 171—185.
null Kmet, L. M., Lee, R. C., & Cook, L. C. (2011). Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Canada: Alberta Heritage Foundation for Medical Research.
null Koong, C. S., & Wu, C. Y The applicability of interactive item templates in varied knowledge types Computers & Education 2011 56 3 781 801 Koong, C. S., & Wu, C. Y. (2011). The applicability of interactive item templates in varied knowledge types. Computers & Education, 56(3), 781—801.
null Kulik, J. A. (2003). Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say. Arlington, VA: sri International.
null Kulik, J. A., Kulik, C. L. C., & Bangert-Drowns, R. L Effectiveness of computer-based education in elementary schools Computers in human behavior 1985 1 1 59 74 Kulik, J. A., Kulik, C. L. C., & Bangert-Drowns, R. L. (1985). Effectiveness of computer-based education in elementary schools. Computers in human behavior, 1(1), 59—74.
null Lamb, R. L., Annetta, L., Firestone, J., & Etopio, E A meta-analysis with examination of moderators of student cognition, affect, and learning outcomes while using serious educational games, serious games, and simulations Computers in Human Behavior 2018 80 158 167 Lamb, R. L., Annetta, L., Firestone, J., & Etopio, E. (2018). A meta-analysis with examination of moderators of student cognition, affect, and learning outcomes while using serious educational games, serious games, and simulations. Computers in Human Behavior, 80, 158—167.
null Leonard, J., Davis, J. E., & Sidler, J. L Cultural relevance and computer-assisted instruction Journal of research on Technology in Education 2005 37 3 263 284 Leonard, J., Davis, J. E., & Sidler, J. L. (2005). Cultural relevance and computer-assisted instruction. Journal of research on Technology in Education, 37(3), 263—284.
null Li, Q., & Ma, X A meta-analysis of the effects of computer technology on school students’ mathematics learning Educational Psychology Review 2010 22 3 215 243 Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215—243.
null Lim, D. H., & Kim, H Motivation and learner characteristics affecting online learning and learning application Journal of Educational Technology Systems 2003 31 4 423 439 Lim, D. H., & Kim, H. (2003). Motivation and learner characteristics affecting online learning and learning application. Journal of Educational Technology Systems, 31(4), 423—439.
null Lou, Y., Abrami, P. C., & d’Apollonia, S Small group and individual learning with technology: A meta-analysis Review of educational research 2001 71 3 449 521 Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of educational research, 71(3), 449—521.
null Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. Intelligent tutoring systems and learning outcomes: A meta-analysis Journal of educational psychology 2014 106 4 901 Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of educational psychology, 106(4), 901.
null Michaelowa, K Primary education quality in francophone Sub-Saharan Africa: Determinants of learning achievement and efficiency considerations World Development 2001 29 10 1699 1716 Michaelowa, K. (2001). Primary education quality in francophone Sub-Saharan Africa: Determinants of learning achievement and efficiency considerations. World Development, 29(10), 1699—1716.
null Moussa, N The importance of learning styles in education Institute for Learning Styles Journal 2014 1 2 19 27 Moussa, N. (2014). The importance of learning styles in education. Institute for Learning Styles Journal, 1(2), 19—27.
null Nattland, A., & Kerres, M. (2009). Computerbasierte Methoden im Unterricht. Karl-Heinz Arnold, Uwe Sandfuchs und Jürgen Wiechmann (Hg.): Handbuch Unterricht. Bad Heilbrunn: Klinkhardt (8423), 317?324.
null Niederhauser, D. S., & Stoddart, T Teachers’ instructional perspectives and use of educational software Teaching and teacher education 2001 17 1 15 31 Niederhauser, D. S., & Stoddart, T. (2001). Teachers’ instructional perspectives and use of educational software. Teaching and teacher education, 17(1), 15—31.
null Oikonomou, V. L., & Patsala, P. (2021). The Integration of Educational Technologies in Foreign Language Education: Teacher Practices and Attitudes in Greece. In IT and the Development of Digital Skills and Competences in Education(pp. 253−275). IGI Global.
null Olaniran, B. A Culture, learning styles, and Web 2 0. Interactive Learning Environments 2009 17 4 261 271 Olaniran, B. A. (2009). Culture, learning styles, and Web 2. 0. Interactive Learning Environments, 17(4), 261—271.
null Ozdemir, M., Sahin, C., Arcagok, S., & Demir, M. K The effect of augmented reality applications in the learning process: A meta-analysis study Eurasian Journal of Educational Research 2018 18 74 165 186 Ozdemir, M., Sahin, C., Arcagok, S., & Demir, M. K. (2018). The effect of augmented reality applications in the learning process: A meta-analysis study. Eurasian Journal of Educational Research, 18(74), 165—186.
null Parmaxi, A., & Zaphiris, P. (2014, June). The evolvement of constructionism: An overview of the literature. In International conference on learning and collaboration technologies (pp. 452−461). Springer, Cham.
null Popescu, E. (2008). Dynamic adaptive hypermedia systems for e-learning (Doctoral dissertation, Université de Technologie de Compiègne).
null Rosenthal, R The file drawer problem and tolerance for null results Psychological bulletin 1979 86 3 638 Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological bulletin, 86(3), 638.
null Sailer, M., Murb?ck, J., & Fischer, F. Digital learning in schools: What does it take beyond digital technology? Teaching and Teacher Education 2021 103 103346 Sailer, M., Murb?ck, J., & Fischer, F. (2021). Digital learning in schools: What does it take beyond digital technology?. Teaching and Teacher Education, 103, 103346.
null S?lj?, R Digital tools and challenges to institutional traditions of learning: technologies, social memory and the performative nature of learning Journal of computer assisted learning 2010 26 1 53 64 S?lj?, R. (2010). Digital tools and challenges to institutional traditions of learning: technologies, social memory and the performative nature of learning. Journal of computer assisted learning, 26(1), 53—64.
null Shashaani, L Gender-differences in computer experience and its influence on computer attitudes Journal of Educational Computing Research 1994 11 4 347 367 Shashaani, L. (1994). Gender-differences in computer experience and its influence on computer attitudes. Journal of Educational Computing Research, 11(4), 347—367.
null Short, M. N., & Uzochukwu, C. (2018). Mobile Technology Integration and Student Learning Outcomes. In Handbook of Research on Mobile Technology, Constructivism, and Meaningful Learning (pp. 178−196). IGI Global.
null Slechtova, P Attitudes of undergraduate students to the use of ICT in education Procedia-Social and Behavioral Sciences 2015 171 1128 1134 Slechtova, P. (2015). Attitudes of undergraduate students to the use of ICT in education. Procedia-Social and Behavioral Sciences, 171, 1128—1134.
null Smetana, L. K., & Bell, R. L. Computer simulations to support science instruction and learning: A critical review of the literature International Journal of Science Education 2012 34 9 1337 1370 Smetana, L. K., & Bell, R. L. (2012). Computer simulations to support science instruction and learning: A critical review of the literature. International Journal of Science Education, 34(9), 1337—1370.
null Soe, K., Koki, S., & Chang, J. M. (2000). Effect of Computer-Assisted Instruction (CAI) on Reading Achievement: A Meta-Analysis.
null Sprenger, D. A., & Schwaninger, A Technology acceptance of four digital learning technologies (classroom response system, classroom chat, e-lectures, and mobile virtual reality) after three months’ usage International Journal of Educational Technology in Higher Education 2021 18 1 1 17 Sprenger, D. A., & Schwaninger, A. (2021). Technology acceptance of four digital learning technologies (classroom response system, classroom chat, e-lectures, and mobile virtual reality) after three months’ usage. International Journal of Educational Technology in Higher Education, 18(1), 1—17.
null Steenbergen-Hu, S., & Cooper, H A meta-analysis of the effectiveness of intelligent tutoring systems on college students' academic learning Journal of Educational Psychology 2014 106 2 331 347 Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students' academic learning. Journal of Educational Psychology, 106(2), 331—347.
null Sung, Y. T., Chang, K. E., & Liu, T. C The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis Computers & Education 2016 94 252 275 Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252—275.
null Taber, K. S. (2017). The role of new educational technology in teaching and learning: A constructivist perspective on digital learning. In Handbook on digital learning for k-12 schools (pp. 397−412). Springer, Cham.
null Tingir, S., Cavlazoglu, B., Caliskan, O., Koklu, O., & Intepe‐Tingir, S Effects of mobile devices on K–12 students' achievement: A meta‐analysis Journal of Computer Assisted Learning 2017 33 4 355 369 Tingir, S., Cavlazoglu, B., Caliskan, O., Koklu, O., & Intepe‐Tingir, S. (2017). Effects of mobile devices on K–12 students' achievement: A meta‐analysis. Journal of Computer Assisted Learning, 33(4), 355—369.
null Usher E. L. (2012) Self-Efficacy for Self-Regulated Learning. In: Seel N. M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA.
null Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M Computer gaming and interactive simulations for learning: A meta-analysis Journal of Educational Computing Research 2006 34 3 229 243 Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229—243.
null Wikan, G., & Molster, T Norwegian secondary school teachers and ICT European Journal of Teacher Education 2011 34 2 209 218 Wikan, G., & Molster, T. (2011). Norwegian secondary school teachers and ICT. European Journal of Teacher Education, 34(2), 209—218.
null Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D A meta-analysis of the cognitive and motivational effects of serious games Journal of educational psychology 2013 105 2 249 Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of educational psychology, 105(2), 249.
null Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., & Irey, R The effectiveness of intelligent tutoring systems on K‐12 students' reading comprehension: A meta‐analysis British Journal of Educational Technology 2019 50 6 3119 3137 Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., & Irey, R. (2019). The effectiveness of intelligent tutoring systems on K‐12 students' reading comprehension: A meta‐analysis. British Journal of Educational Technology, 50(6), 3119—3137.
Outlines

/