Strengthen Replication of Educational Quantitative Research in China

  • Yi Li ,
  • Jianqing Zhang ,
  • Yanling Yang
Expand
  • 1. Faculty of Education, Southwest University, Chongqing 400715, China
    2. Collaborative Innovation Center of Assessment Toward Basic Education Quality Southwest University Branch, Chongqing 400715, China
    3. Basic Education Research Center of Southwest University, Chongqing 400715, China
    4. Primary School Attached to Southwest University, Chongqing 400700, China
    5. Zhonghua Road Primary School, Yuzhong District, Chongqing 400010, China

Accepted date: 2022-05-16

  Online published: 2023-10-27

Abstract

In this study, the principle of replication, replication of research and repeated research were discriminated theoretically. Then it discussed three types of repeated research (exact, direct and conceptual) and two functions (validation and extension) and elaborated the meaning and value of replication. Based on that, this study took the repeated quantitative research published in 37 educational CSSCI journals from 2015 to 2020 as samples to analyze the current situation and the problems of replication for the educational quantitative research in China. The study found that it is not fully recognized that replication is the basic position of innovative research; the undeveloped foundation of quantitative research leads to the difficulty in conducting repeated researches; the failure of replication is misunderstood, which leads to the panic of ‘replication crisis’; the traditional thinking inertia hinders the original research teams to continue their original research interest. Accordingly, the study presented the following suggestions: reinforce understanding of the value of repeated research, establish incentive mechanism of repeated research; strengthen training of researchers in quantitative research methods, build an open science platform to turn the ‘black box’ into the ‘white box; overcome the stereotype of “success rate only”, build a stable research team to follow up and form research clusters.

Cite this article

Yi Li , Jianqing Zhang , Yanling Yang . Strengthen Replication of Educational Quantitative Research in China[J]. Journal of East China Normal University(Educational Sciences), 2023 , 41(11) : 85 -96 . DOI: 10.16382/j.cnki.1000-5560.2023.11.007

References

null 波普尔. (1963). 猜想与反驳(傅季重, 纪树立, 周昌忠, 蒋戈为译). 上海: 上海译文出版社.
null 程亮. 教育研究原创性缺乏: 现象与原因 教育理论与实践 2004 15 9 12 程亮. (2004). 教育研究原创性缺乏: 现象与原因. 教育理论与实践,(15),9—12.
null 范涌峰, 宋乃庆. 教育研究科学化: 限度与突破 教育研究 2016 01 94 101 范涌峰, 宋乃庆. (2016). 教育研究科学化: 限度与突破. 教育研究,(01),94—101.
null 哈贝马斯. (1999). 认识与兴趣(郭官义, 李黎译). 上海: 学林出版社.
null 韩双淼, 谢静. 国外教育研究方法的应用特征——基于2000-2019年34本教育学SSCI收录期刊的文献计量分析 高等教育研究 2021 01 68 76 韩双淼, 谢静. (2021). 国外教育研究方法的应用特征—基于2000-2019年34本教育学SSCI收录期刊的文献计量分析. 高等教育研究,(01),68—76.
null 何克抗. 创立中国特色教育科学实证研究的理论方法体系 电化教育研究 2016 02 5 9+17 何克抗. (2016). 创立中国特色教育科学实证研究的理论方法体系. 电化教育研究,(02),5—9+17.
null 吕晶. 中国教育实证研究中的定量方法: 五年应用述评 华东师范大学学报 (教育科学版) 2020 38 9 36 55 吕晶. (2020). 中国教育实证研究中的定量方法: 五年应用述评. 华东师范大学学报 (教育科学版),38(9),36—55.
null 李琳璐. 教育研究范式的祛魅: 思辨与实证的融合共生 大学教育科学 2021 03 31 38 李琳璐. (2021). 教育研究范式的祛魅: 思辨与实证的融合共生. 大学教育科学,(03),31—38.
null 任思腾. 科学实验中的可重复概念 自然辩证法通讯 2020 42 09 46 52 任思腾. (2020). 科学实验中的可重复概念. 自然辩证法通讯,42(09),46—52.
null 沙沃森, 汤. (2006). 教育的科学研究(曹晓南等译). 北京: 教育科学出版社.
null 王晓诚, 贾丽娜, 金元英. 家庭社会经济地位对初中生阅读成就的影响: 阅读动机和阅读活动的中介作用 心理与行为研究 2020 06 839 845 王晓诚, 贾丽娜, 金元英. (2020). 家庭社会经济地位对初中生阅读成就的影响: 阅读动机和阅读活动的中介作用. 心理与行为研究,(06),839—845.
null 阎光才. 教育及社会科学研究中的数据——兼议当前的大数据热潮 北京大学教育评论 2013 11 04 77 86 阎光才. (2013). 教育及社会科学研究中的数据—兼议当前的大数据热潮. 北京大学教育评论,11(04),77—86.
null 姚计海. 教育实证研究方法的范式问题与反思 华东师范大学学报(教育科学版) 2017 03 64 71+169?170 姚计海. (2017). 教育实证研究方法的范式问题与反思. 华东师范大学学报(教育科学版),(03),64—71+169?170.
null 袁振国. 实证研究是教育学走向科学的必要途径 华东师范大学学报(教育科学版) 2017 35 03 4 17+168 袁振国. (2017). 实证研究是教育学走向科学的必要途径. 华东师范大学学报(教育科学版),35(03),4—17+168.
null 张国平, 高耀明. 美国大学高等教育学专业研究生培养的特色及其启示 高等教育研究 2019 9 53 60 张国平, 高耀明. (2019). 美国大学高等教育学专业研究生培养的特色及其启示. 高等教育研究,9,53—60.
null 周明. 教育实证研究的地位与价值——中国教育学会教育理论刊物分会2015年年会综述 教育科学 2016 02 93 96 周明. (2016). 教育实证研究的地位与价值—中国教育学会教育理论刊物分会2015年年会综述. 教育科学,(02),93—96.
null 朱军文, 马银琦. 教育实证研究这五年: 特征、趋势及展望 华东师范大学学报(教育科学版) 2020 09 16 35 朱军文, 马银琦. (2020). 教育实证研究这五年: 特征、趋势及展望. 华东师范大学学报(教育科学版),(09),16—35.
null Ajzen, I. The theory of planned behavior Organizational Behavior & Human Decision Processes 1991 50 2 179 211 Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50(2), 179—211.
null Bradley, R. H. & Corwyn, R. F. Socioeconomic status and child development Annual Review of Psychology 2002 53 371 399 Bradley, R. H. & Corwyn, R. F. (2002). Socioeconomic status and child development. Annual Review of Psychology, 53, 371—399.
null Briggs, C. & Elkind, D. Characteristics of early readers Perceptual & Motor Skills 1977 44 3c 1231 1237 Briggs, C. & Elkind, D. (1977). Characteristics of early readers. Perceptual & Motor Skills, 44(3c), 1231—1237.
null Capraro, R. M. & Thompson, B. (2008). The educational researcher defined: what will future researchers be trained to do?. The Journal of Educational Research, 101(4), 247?253.
null Chen, C. P. Understanding mobile English-learning gaming adopters in the self-learning market: The uses and gratification expectancy model Computers & Education 2018 126 217 230 Chen, C. P. (2018). Understanding mobile English-learning gaming adopters in the self-learning market: The uses and gratification expectancy model. Computers & Education, 126, 217—230.
null Chhin, C. S., Taylor, K. A., & Wei, W. S. Supporting a culture of replication: An examination of education and special education research grants funded by the Institute of Education Sciences Educational Researcher 2018 47 9 594 605 Chhin, C. S., Taylor, K. A., & Wei, W. S. (2018). Supporting a culture of replication: An examination of education and special education research grants funded by the Institute of Education Sciences. Educational Researcher, 47(9), 594—605.
null Dear, P. (2001). Revolutionizing the sciences: European knowledge and its ambitions, 1500–1700. New York: Palgrave Macmillan.
null Durkin, D. (1966). Children who read early. New York: Teachers College Press.
null Enserink & Martin. Controversial studies give a deadly flu virus wings Science 2011 334 6060 1192 3 Enserink & Martin. (2011). Controversial studies give a deadly flu virus wings. Science, 334(6060), 1192—3.
null Fishbein, M., Ajzen, I. & Belief, A. Belief, attitude, intention, and behavior: An introduction to theory and research Contemporary Sociology 1977 6 2 244 245 Fishbein, M., Ajzen, I. & Belief, A. (1977). Belief, attitude, intention, and behavior: An introduction to theory and research. Contemporary Sociology, 6(2), 244—245.
null Francis, G. The psychology of replication and replication in psychology Perspectives on Psychological Science. A Journal of the Association for Psychological Science 2012 7 6 585 Francis, G. (2012). The psychology of replication and replication in psychology. Perspectives on Psychological Science. A Journal of the Association for Psychological Science, 7(6), 585.
null Gelman, A. Don’t characterize replications as successes or failures The Behavioral and Brain Sciences 2018 41 128 20 Gelman, A. (2018). Don’t characterize replications as successes or failures. The Behavioral and Brain Sciences, 41, 128—20.
null Gewin, V. Data sharing: An open mind on open data Nature 2016 529 7584 117 119 Gewin, V. (2016). Data sharing: An open mind on open data. Nature, 529(7584), 117—119.
null Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. Comment on “Estimating the reproducibility of psychological science” Science 2016 351 6277 1037 Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the reproducibility of psychological science”. Science, 351(6277), 1037.
null Harwell, M., Maeda, Y., Bishop, K. & Xie, A. The surprisingly modest relationship between SES and educational achievement The Journal of Experimental Education 2017 85 2 197 214 Harwell, M., Maeda, Y., Bishop, K. & Xie, A. (2017). The surprisingly modest relationship between SES and educational achievement. The Journal of Experimental Education, 85(2), 197—214.
null Kier, M. W., Blanchard, M. R., Osborne, J. W. & Albert, J. L. The development of the STEM career interest survey (STEM-CIS) Research in Science Education 2014 44 3 461 481 Kier, M. W., Blanchard, M. R., Osborne, J. W. & Albert, J. L. (2014). The development of the STEM career interest survey (STEM-CIS). Research in Science Education, 44(3), 461—481.
null Koyunlu Unlu, Z., Dokme, I., & Unlu, V. Adaptation of the science, technology, engineering, and mathematics career interest survey (STEM-CIS) into Turkish Eurasian Journal of Educational Research 2016 16 63 21 36 Koyunlu Unlu, Z., Dokme, I., & Unlu, V. (2016). Adaptation of the science, technology, engineering, and mathematics career interest survey (STEM-CIS) into Turkish. Eurasian Journal of Educational Research, 16(63), 21—36.
null Levy, L. H. Reflections on replications and the experimenter bias effect Journal of Consulting and Clinical Psychology 1969 33 15 17 Levy, L. H. (1969). Reflections on replications and the experimenter bias effect. Journal of Consulting and Clinical Psychology, 33, 15—17.
null Light, R. J. Measures of response agreement for qualitative data: some generalizations and alternatives Psychological Bulletin 1971 76 5 365 Light, R. J. (1971). Measures of response agreement for qualitative data: some generalizations and alternatives. Psychological Bulletin, 76(5), 365.
null Madden, C. S., Easley, R. W. & Dunn, M. G. How Journal Editors View Replication Research Journal of Advertising 1995 24 4 77 87 Madden, C. S., Easley, R. W. & Dunn, M. G. (1995). How Journal Editors View Replication Research. Journal of Advertising, 24(4), 77—87.
null Mahoney, M. J. Open exchange and epistemic progress American Psychologist 1985 40 1 29 39 Mahoney, M. J. (1985). Open exchange and epistemic progress. American Psychologist, 40(1), 29—39.
null Makel, M. C., Plucker, J. A. & Hegarty, B. Replications in psychology research: how often do they really occur? Perspectives on Psychological Science A Journal of the Association for Psychological Science 2012 7 6 537 Makel, M. C., Plucker, J. A. & Hegarty, B. (2012). Replications in psychology research: how often do they really occur?. Perspectives on Psychological Science A Journal of the Association for Psychological Science, 7(6), 537.
null Makel, M. C. & Plucker, J. A. Facts are more important than novelty: Replication in the education sciences Educational Researcher 2014 43 6 304 316 Makel, M. C. & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304—316.
null Makel, M. C., Plucker, J. A., Freeman, J., Lombardi, A., Simonsen, B. & Coyne, M. Replication of special education research: Necessary but far too rare Remedial and Special Education 2016 31 2 48 53 Makel, M. C., Plucker, J. A., Freeman, J., Lombardi, A., Simonsen, B. & Coyne, M. (2016). Replication of special education research: Necessary but far too rare. Remedial and Special Education, 31(2), 48—53.
null Plucker, Jonathan. & Makel, Matthew. (2021). Replication is important for educational psychology: Recent developments and key issues. Educational Psychologist. 5(6), 1?11.
null Monya, B. 1, 500 scientists lift the lid on reproducibility Nature 2016 533 7604 452 454 Monya, B. (2016). 1, 500 scientists lift the lid on reproducibility. Nature, 533(7604), 452—454.
null Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Yarkoni, T. Promoting an open research culture Science 2015 348 6242 1422 1425 Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422—1425.
null Ostrom, T. H. To replicate or explicate American Psychologist 1971 26 3 312 Ostrom, T. H. (1971). To replicate or explicate. American Psychologist, 26(3), 312.
null Pashler, H. & Harris, C. R. Is the replicability crisis overblown? Three arguments examined Perspectives in Psychological Science 2012 7 6 531 Pashler, H. & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives in Psychological Science, 7(6), 531.
null Ritchie, S. J., Wiseman, R. & French, C. C. Replication, replication, replication Psychologist 2012 25 346 348 Ritchie, S. J., Wiseman, R. & French, C. C. (2012). Replication, replication, replication. Psychologist, 25, 346—348.
null Rosenthal, R. & Fode, K. L. The effect of experimenter bias on the performance of the albino rat Behavioral Science 1963 8 3 183 189 Rosenthal, R. & Fode, K. L. (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science, 8(3), 183—189.
null Rosenthal, R. & Jacobson, L. Pygmalion in the classroom Urban Review 1968 3 1 16 20 Rosenthal, R. & Jacobson, L. (1968). Pygmalion in the classroom. Urban Review, 3(1), 16—20.
null Schmidt, S. Shall we really do it again? The powerful concept of replication is neglected in the social sciences Review of General Psychology 2009 13 2 90 100 Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90—100.
null Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant Psychological Science 2011 22 11 1359 1366 Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359—1366.
null Sirin, S. R. Socioeconomic status and academic achievement: A meta-analytic review of research Review of Educational Research 2005 75 3 417 453 Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417—453.
null Towne, L., Fletcher, J. M., Wise, L. L. & Winters, T. M. Advancing scientific research in education National Academies Press 2005 106 100 329 337 Towne, L., Fletcher, J. M., Wise, L. L. & Winters, T. M. (2005). Advancing scientific research in education. National Academies Press, 106(100), 329—337.
null Travers, J. C., Cook, B. G., Therrien, W. J. & MD Coyne. Replication research and special education Remedial & Special Education 2016 37 4 195 204 Travers, J. C., Cook, B. G., Therrien, W. J. & MD Coyne. (2016). Replication research and special education. Remedial & Special Education, 37(4), 195—204.
null Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. Contextual sensitivity in scientific reproducibility Proceedings of the National Academy of Sciences of the United States of America 2016 113 23 6454 6459 Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016). Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences of the United States of America, 113(23), 6454—6459.
null Wentzel, K. R. Open science reforms: Strengths, challenges, and future directions Educational Psychologist 2021 56 2 161 173 Wentzel, K. R. (2021). Open science reforms: Strengths, challenges, and future directions. Educational Psychologist, 56(2), 161—173.
Outlines

/