寻找“最佳证据”：如何运用元分析进行文献综述——以STEM教育对学生成绩的影响研究为例

doi:10.16382/j.cnki.1000-5560.2020.06.005

摘要/Abstract

摘要： 元分析相对于传统的主观性文献综述方法而言，能够更加有效、客观和规范地从既有实证研究文献中梳理出一般性、规律性的结论，因此成为教育循证改革过程中寻找"最佳证据"的重要方法。与国外相比，我国运用元分析方法进行实证文献综述的研究还比较少。针对于此，本研究以STEM教育对学生学业成绩的影响为例，展示了运用元分析方法进行实证研究综述的过程。研究发现STEM教育有利于提高学生的学业成绩（d=0.410），STEM教育方法、受教育阶段、地区和样本量等因素均会显著影响到STEM教育的效果。这样的研究，不仅提供了关于STEM教育效果的一般性证据，而且在方法层面探讨了如何通过文献综述获得可靠证据以支持教育改革。

关键词: 教育循证改革, STEM教育, 学业成绩, 元分析

Abstract: Compared with the method of traditional subjective literature review, meta-analysis is more objective, normative, and can effectively sort out general conclusion from existing empirical researches. Therefore, it has become an important method to identify the “best evidence” in the process of evidence-based educational reform. So far there are few educational studies in China using the meta-analysis method to review the empirical literatures. In view of this, this research analyzes the impact of STEM education on students’ academic achievement as an example of how to use meta-analysis to synthesise existing empirical research. We found: 1) STEM education is conductive to students academic achievements (d=0.410); 2) factors like educational methods, students’ educational stage, location and sample size can affect the effects. The study provides the general evidence of effects of STEM education and explores how to obtain reliable evidence through a literature review to support educational reform.

Key words: evidence-based educational reform, STEM education, academic achievement, meta-analysis

曾昭炳, 姚继军. 寻找“最佳证据”：如何运用元分析进行文献综述——以STEM教育对学生成绩的影响研究为例[J]. 华东师范大学学报(教育科学版), 2020, 38(6): 70-85.

Zeng Zhaobing, Yao Jijun. Identifying the “Best Evidence”: How to Use Meta-analysis to Conduct a Literature Review—A Case of STEM Education’s Effect on Students’ Academic Achievement[J]. Journal of East China Normal University(Educational Sciences), 2020, 38(6): 70-85.

参考文献

卢谢峰, 唐源鸿, 曾凡梅. (2011). 效应量: 估计、报告和解释. 心理学探新,31(3),260-264
姚计海. (2017). “文献法”是研究方法吗?——兼谈研究整合法. 国家教育行政学院学报,19(7),89-94
张斌贤, 李曙光. (2015). 文献综述与教育学博士学位论文撰写. 学位与研究生教育,32(1),59-63
张天嵩, 董圣杰, 周支瑞. (2015). 高级Meta分析方法——基于Stata实现. 上海: 复旦大学出版社.
Acar, D., Tertemiz, N., & Tasdemir, A. (2018). The Effects of STEM Training on the Academic Achievement of 4th Graders in Science and Mathematics and their Views on STEM Training Teachers. International Electronic Journal of Elementary Education, 10(4), 505-513
Akinoglu, O., & Tandogan, R. O. (2007). The Effects of Problem-Based Active Learning in Science Education on Students’ Academic Achievement, Attitude and Concept Learning. Eurasia Journal of Mathematics, Science & Technology Education, 3(1), 71-81
An, D. (2012). A Meta-Analysis of the Effectiveness of STEM-Programs in the United States. Seoul: Sangmyung University.
Apugliese, A., & Lewis, S. T. (2017). Impact of instructional decisions on the effectiveness of cooperative learning in chemistry through meta-analysis. Chemistry Education Research and Practice, 18, 271-278
Araz, G. (2007). The Effect of Problem-Based Learning on the Elementary School Students' Achievement in Genetics. Department of Elementary Science and Mathematics Education.
Atkinson, R. D., & Mayo, M. J. (2010). Refueling the US Innovation Economy: Fresh Approaches to Science, Technology, Engineering and Mathematics (STEM) Education. The Information Technology & Innovation Foundation.
Barth, K. N. (2013). An Investigation of the Effects of Integrating Science and Engineering Content and Pedagogy in an Elementary School Classroom (Doctoral dissertation). Provo: Brigham Young University.
Becker, K., & Park, K. (2011). Integrative Approaches among Science, Technology, Engineering, and Mathematics (STEM) Subjects on Students' Learning: A Meta-Analysis. Journal of STEM Education: Innovations and Research, 12(5), 23-37
Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., & Green, C. S. (2018). Meta- Analysis of Action Video Game Impact on Perceptual, Attentional, and Cognitive Skills. Psychological Bulletin, 144(1), 77-110
Belland, B. R., Walker, A. E., Olsen, M. W., & Leary, H. (2015). A Pilot Meta-Analysis of Computer-Based Scaffolding in STEM Education. Educational Technology & Society, 18(1), 183-197
Belland, B. R., Walker, A. E., & Kim, N. J. (2017). A Bayesian Network Meta-Analysis to Synthesize the Influence of Contexts of Scaffolding Use on Cognitive Outcomes in STEM Education. Review of Educational Research, 87(6), 1042-1081
Bicer, A., Navruz, B., Capraro, R. M., Capraro, M. M., Oner, T., & Boedeker, P. (2015). STEM Schools VS. Non-STEM Schools: Comparing Students’ Mathematics Growth Rate on High-Stakes Test Performance. International Journal on New Trends in Education and Their Implications, 6(1), 138-150
Bicer, A., Capraro, R. M., & Capraro, M. M. (2017). Integrated STEM Assessment Model. EURASIA Journal of Mathematics, Science & Technology Education, 13(7), 3959-3968
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. New Jersey: John Wiley & Sons, Ltd.
Cakici, Y., & Turkmen, N. (2013). An Investigation of the Effect of Project-Based Learning Approach on Children’s Achievement and Attitude in Science. The Online Journal of Science and Technology, 3(2), 10-17
Carbonneau, K. J., Marley, S. C., Selig, J. P. (2013). A Meta-Analysis of the Efficacy of Teaching Mathematics With Concrete Manipulatives. Journal of Educational Psychology, 105(2), 380-400
Carmichael, C. C. (2017). A State-By-State Policy Analysis of STEM Education for K-12 Public Schools (Ed.D. Dissertation). New Jersey: Seton Hall University.
Cervetii, G. N., Barber, J., Dorph, R., Pearson, D., & Goldschmidt, P. G. (2012). The Impact of an Integrated Approach to Science and Literacy in Elementary School Classrooms. Journal of Research in Science Teaching, 19(5), 631-658
Cheung, A., & Slavin, R. E. (2012). Effects of Educational Technology Applications on Reading Outcomes for Struggling Readers: A Best Evidence Synthesis. Center for Research and Reform in Education, Johns Hopkins University.
Cheung, A., & Slavin, R. E. (2013a). The Effectiveness of Educational Technology Applications on Mathematics Achievement in K-12 Classrooms: A meta-analysis. Educational Research Review, 9(1), 88-11
Cheung, A. C. K., & Slavin, R. E. (2013b). Effects of Educational Technology Applications on Reading Outcomes for Struggling Readers: A Best Evidence Synthesis. Reading Research Quarterly, 48(3), 277-299
Cheung, A., & Slavin, R. E. (2016). How Methodological Features Affect Effect Sizes in Education. Educational Researcher, 45(5), 283-292
Chow, S. L. (1988). Significance Test or Effect Size?. Psychological Bulletin, 103(1), 105-110
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Cotabish, A., Dailey, D., Robinson, A., & Hughes, G. (2013). The Effects of a STEM Intervention on Elementary Students’ Science Knowledge and Skills. School Science and Mathematics, 113(5), 215-226
Denney, A. S., Tewksbury, R. (2013). How to Write a Literature Review. Journal of Criminal Justice Education, 24(2), 218-234
Duval, S., & Tweedie, R. (2000). Trim and Fill: A Simple Funnel-Plot-Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis. Biometrics, 56(2), 455-463
Fan, S. C., & Yu, K. C. (2015). How an integrative STEM curriculum can benefit students in engineering design practices. International Journal of Technology & Design Education, 27(1), 1-23
Fatade, A. O., Mogari, D., & Arigbabu, A. A. (2013). Effect of Problem-Based Learning on Senior Secondary School Students’ Achievements in Further Mathematics. Acta Didactica Napocensia, 6(3), 27-44
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active Learning Increases Student Performance in Science, Engineering, and Mathematics. National Acad Science, 111(23), 8410-8415
Han, S., Rosli, R., Capraro, M. M., & Capraro, R. M. (2016). The Effect of Science, Technology, Engineering and Mathematics (STEM) Project Based Learning (PBL) on Students’ Achievement in Four Mathematics Topics. Journal of Turkish Science Education, 13, 3-29
Harris, C. J., Penuel, W. R., D’Angelo, C. M., DeBarger, A. H., Gallagher, L.P., Kennedy, C. A., Cheng, B. H., & Krajcik, J. S. (2015). Impact of Project-Based Curriculum Materials on Student Learning in Science: Results of a Randomized Controlled Trial. Journal of Research in Science Teaching, 52(10), 1362-1385
Holveck, S. E. (2012). Teaching for Conceptual Change in a Density Unit Taught to 7th Graders: Comparing Two Teaching Methodologies-Scientific Inquiry and a Traditional Approach. Oregon: University of Oregon.
Honey, M., Pearson, G., & Schweingruber, H. (2014). STEM Integration in K-12 Education Status, Prospects, and an Agenda for Research. Washington D. C.: National Academies Press.
James, J. S. (2014). Science, Technology, Engineering, and Mathematics (STEM) Curriculum and Seventh Grade Mathematics and Science Achievement. Michigan: ProQuest LLC.
Kalaian, S. A., & Kasim, R. M. (2014). A Meta-Analytic Review of Studies of the Effectiveness of Small-Group Learning Methods on Statistics Achievement. Journal of Statistics Education, 22(1), 1-20
Kala. (2018). Effectiveness of Small-Group Learning Pedagogies in Engineering and Technology Education: A Meta-Analysis. Journal of Technology Education, 29(2), 20-35
Kassir, H. A. N. (2013). The effectiveness of the Science-Inquiry Teaching Approach on the Students’ Achievement and Engagement in the UAE’ Public Schools. Dubai: The British University in Dubai.
Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3, 1-11
Kim, K. H., Tassel-Baska, J. V., Bracken, B. A., Feng, A., Stambaugh, T., & Bland, L. (2012). Project Clarion: Three Years of Science Instruction in Title I Schools among K-Third Grade Students. Research in Science Education, 42(5), 813-829
Kim, N. A., Belland, B. R., & Walker, A. E. (2018). Effectiveness of Computer-Based Scaffolding in the Context of Problem-Based Learning for Stem Education: Bayesian Meta-analysis. Educational Psychology Review, 30(2), 397-429
Kine, M. S. (2017). Visual-Spatial Ability in STEM Education: Transforming Research into Practice. Switzerland: Springer International Publishing Switzerland, 2017.
Kizkapan, O., & Bektas, O. (2017). The Effect of Project Based Learning on Seventh Grade Students’ Academic Achievement. International Journal of Instruction, 10(1), 37-54
Korur, F., Efe, G., Erdogan, F., & Tunc, B. (2015). Effects of Toy Crane Design-Based Learning on Simple Machines. International Journal of Science and Mathematics Education, 15, 251-271
Lazowski, R., & Hulleman, C. (2015). Motivation Interventions in Education: A Meta-Analytic Review. Review of Educational Research, 86(2), 1-39
Li, Y., Huang, Z., Jiang, M., & Chang, T. W. (2016). The Effect on Pupils’ Science Performance and Problem-Solving Ability through Lego: An Engineering Design-based Modeling Approach. Educational Technology & Society, 19(3), 143-156
Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-Analysis. Thousand Oaks: Sage Publications, Inc.
Maxwell, D. O., Lambeth, D. T., & Cox, J. T. (2015). Effects of Using Inquiry-Based Learning on Science Achievement for Fifth-Grade Students. Asia-Pacific Forum on Science Learning and Teaching, 16(1), 1-31
McDonald, C. V. (2016). STEM Education: A Review of the Contribution of the Disciplines of Science, Technology, Engineering and Mathematics. Science Education International, 27(4), 530-569
Merrill, C. (2001). Integrated Technology, Mathematics, and Science Education: A Quasi-Experiment. Journal of Industrial Teacher Education, 38, 45-61
Michko, G. M. (2008). Meta-Analysis of Effectiveness of Technology Use in Undergraduate Engineering Education. https://www.researchgate.net/publication/224361059_Meta-analysis_of_effectiveness_of_technology_use_in_undergraduate_engineering_education.
Ojaleye, O., & Awofala, A. O. A. (2018). Blended Learning and Problem-Based Learning Instructional Strategies as Determinants of Senior Secondary School Students’ Achievement in Algebra. International Journal of Research in Education and Science, 4(2), 486-501
Olivarez, N. (2013). The Impact of a STEM Program on Academic Achievement of Eighth Grade Students in a South Texas Middle School. Corpus Christi, Texas: Texas A & M University - Corpus Christi.
Rehmat, A. P. (2015). Engineering the Path to Higher-Order Thinking in Elementary Education: A Problem-Based Learning Approach for STEM Integration. Las Vegas: University of Nevada.
Robinson, A., Dailey, D., Hughes, G., & Cotabish, A. (2014). The Effects of a Science- Focused STEM Intervention on Gifted Elementary Students’ Science Knowledge and Skills. Journal of Advanced Academics, 25(3), 189-213
Rosenthal, R. (1979). The “File Drawer Problem” and Tolerance for Null Results. Psychol Bull, 86, 638-641
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication Bias in Meta-Analysis Prevention, Assessment and Adjustments.Chichester: John Wiley & Sons Ltd.
Rozek, C. S., Ramirez, G., Fine, R. D., & Beilock, S. L. (2019). Reducing Socioeconomic Disparities in the STEM Pipeline Through Student Emotion Regulation. Proceeding of the National Academy of Sciences of the United States of America, 116(5), 1553-1558
Sanders, M. (2009). STEM, STEM Education, STEMmania. Technology Teacher, 68(4), 20-26
Sarac, H. (2018). The Effect of Science, Technology, Engineering and Mathematics-Stem Education Practices on Students’ Learning Outcomes: A meta-analysis. The Turkey Online Journal of Educational Technology, 17(2), 125-142
Slavin, R. E. (2008). Evidence-Based Reform in Education: What Will It Take?. European Educational Research Journal, 7(1), 124-128
Slavin, R. E. (2017). Evidence-Based Reform in Education. Journal of Education for Students Placed at Risk, 22(3), 178-184
Sokolowski, A. (2013). The Effects of Exploratory Learning Environments on Students’ Mathematics Achievement. Texas: Texas A&M University.
Sokolowski, A., Li, P., & Willson, V. (2015). The Effects of Using Exploratory Computerized Environments in Grades 1 to 8 Mathematics: A Meta-Analysis of Research. International Journal of STEM Education, 2(8), 1-17
Steenbergen-Hu, S., & Cooper, H. (2013). A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on K-12 Students’ Mathematical Learning. Journal of Educational Psychology, 105(4), 970-987
Taylor, K. (2016). Collaborative Robotics, More Than Just Working in Groups: Effects of Student Collaboration on Learning Motivation, Collaborative Problem Solving, and Science Process Skills in Robotic Activities. Boise: Boise State University.
Valentine, J. C., & Cooper, H. (2003). What Works Clearinghouse Study Design and Implementation Assessment Device (Version 0.6). Washington, DC: U.S. Department of Education.
Yasin, R. M., & Yunus, N. S. (2014). A Meta-Analysis Study on the Effectiveness of Creativity Approaches in Technology and Engineering Education. Asian Social Science, 10(3), 242-252
Yildirim, B. (2016). An Analyses and Meta-Synthesis of Research on STEM Education. Journal of Education and Practice, 7(34), 23-33
Young, J., Ortiz, N., & Young, J. (2017). STEMulating Interest: A Meta-Analysis of the Effects of Out-of-School Time on Student STEM Interest. International Journal of Education in Mathematics, Science and Technology, 5(1), 62-74
Zeng, Z., Yao, J. Gu, H., & Przybylski, R. (2018). A Meta-Analysis on the Effects of STEM Education on Students’ Abilities. Science Insights Education Frontiers, 1(1), 3-16

[1]	张静. 智商与情商哪个对学习更重要？[J]. 华东师范大学学报(教育科学版), 2022, 40(11): 69-79.
[2]	姜怡. 高中生学习动机发展模态及其对学业成绩的影响[J]. 华东师范大学学报(教育科学版), 2022, 40(11): 80-91.
[3]	雷浩, 李雪. 数字工具支持的教学对学生学习结果有何影响？——来自137项实验与准实验的元分析证据[J]. 华东师范大学学报(教育科学版), 2022, 40(11): 92-109.
[4]	成刚, 杜思慧, 余倩. “望子成龙”有效吗？——基于亲子教育期望偏差对学业成绩的影响研究[J]. 华东师范大学学报(教育科学版), 2022, 40(1): 74-87.
[5]	李佳哲, 胡咏梅. 家长学习参与和中小学生学业成绩的关系研究——基于亲子关系和学习自信心的有中介的调节模型分析[J]. 华东师范大学学报(教育科学版), 2021, 39(7): 72-83.
[6]	魏易, 罗滨, 林秀艳, 杨智君, 王梦. 区域教研对学生学业成绩影响的实证研究——以北京市海淀区为例[J]. 华东师范大学学报(教育科学版), 2021, 39(5): 12-54.
[7]	[美]罗伯特•斯莱文, 张志强, 庄腾腾. 证据驱动的教育改革如何推动教育发展[J]. 华东师范大学学报(教育科学版), 2021, 39(3): 14-22.
[8]	孙立会, 胡琳琳. 编程真的能促进儿童的个体发展吗？——基于28 项实验和准实验研究的元分析[J]. 华东师范大学学报(教育科学版), 2021, 39(11): 45-58.
[9]	胡咏梅, 元静. 学校投入与家庭投入哪个更重要？——回应由《科尔曼报告》引起的关于学校与家庭作用之争[J]. 华东师范大学学报(教育科学版), 2021, 39(1): 1-25.
[10]	刘坚, 赵利曼, 杜宵丰, 徐冠兴. 高中生睡眠时间与高学业成绩的理想匹配模式探究及预警[J]. 华东师范大学学报(教育科学版), 2020, 38(3): 71-79.
[11]	唐一鹏, 王闯, 胡咏梅. 如何提升中小学生的学业成绩？——基于学习策略与教学策略改进的视角[J]. 华东师范大学学报(教育科学版), 2020, 38(3): 93-105.
[12]	赵晨, 陈思, 曹艳, [美]凯瑟琳·斯诺, 卢迈. 教育精准扶贫：“一村一园”计划对农村儿童学业成绩的长效影响研究[J]. 华东师范大学学报(教育科学版), 2020, 38(2): 114-125.
[13]	大卫·安德森, 季娇. 从STEM教育到STEAM教育——大卫·安德森与季娇关于博物馆教育的对话[J]. 华东师范大学学报(教育科学版), 2017, 35(4): 122-129+139.
[14]	林静. 美国STEM教育质量评价新动向——NAEP技术与工程素养评价要点与启示[J]. 华东师范大学学报(教育科学版), 2017, 35(1): 78-86+122.
[15]	刘莉莉. 中小学校长胜任特征的元分析研究[J]. 华东师范大学学报(教育科学版), 2015, 33(4): 36-40.