Strengthen Replication of Educational Quantitative Research in China

doi:10.16382/j.cnki.1000-5560.2023.11.007

Abstract

Abstract:

In this study, the principle of replication, replication of research and repeated research were discriminated theoretically. Then it discussed three types of repeated research (exact, direct and conceptual) and two functions (validation and extension) and elaborated the meaning and value of replication. Based on that, this study took the repeated quantitative research published in 37 educational CSSCI journals from 2015 to 2020 as samples to analyze the current situation and the problems of replication for the educational quantitative research in China. The study found that it is not fully recognized that replication is the basic position of innovative research; the undeveloped foundation of quantitative research leads to the difficulty in conducting repeated researches; the failure of replication is misunderstood, which leads to the panic of ‘replication crisis’; the traditional thinking inertia hinders the original research teams to continue their original research interest. Accordingly, the study presented the following suggestions: reinforce understanding of the value of repeated research, establish incentive mechanism of repeated research; strengthen training of researchers in quantitative research methods, build an open science platform to turn the ‘black box’ into the ‘white box; overcome the stereotype of “success rate only”, build a stable research team to follow up and form research clusters.

Key words: replication, quantitative research, repeated research, scientization, CSSCI

Yi Li, Jianqing Zhang, Yanling Yang. Strengthen Replication of Educational Quantitative Research in China[J]. Journal of East China Normal University(Educational Sciences), 2023, 41(11): 85-96.

Figures/Tables 6

References 0

	波普尔. (1963). 猜想与反驳(傅季重, 纪树立, 周昌忠, 蒋戈为译). 上海: 上海译文出版社.
	程亮. 教育研究原创性缺乏: 现象与原因教育理论与实践 2004 15 9 12 程亮. (2004). 教育研究原创性缺乏: 现象与原因. 教育理论与实践,(15),9—12.
	范涌峰, 宋乃庆. 教育研究科学化: 限度与突破教育研究 2016 01 94 101 范涌峰, 宋乃庆. (2016). 教育研究科学化: 限度与突破. 教育研究,(01),94—101.
	哈贝马斯. (1999). 认识与兴趣(郭官义, 李黎译). 上海: 学林出版社.
	韩双淼, 谢静. 国外教育研究方法的应用特征——基于2000-2019年34本教育学SSCI收录期刊的文献计量分析高等教育研究 2021 01 68 76 韩双淼, 谢静. (2021). 国外教育研究方法的应用特征—基于2000-2019年34本教育学SSCI收录期刊的文献计量分析. 高等教育研究,(01),68—76.
	何克抗. 创立中国特色教育科学实证研究的理论方法体系电化教育研究 2016 02 5 9+17 何克抗. (2016). 创立中国特色教育科学实证研究的理论方法体系. 电化教育研究,(02),5—9+17.
	吕晶. 中国教育实证研究中的定量方法: 五年应用述评华东师范大学学报 (教育科学版) 2020 38 9 36 55 吕晶. (2020). 中国教育实证研究中的定量方法: 五年应用述评. 华东师范大学学报 (教育科学版),38(9),36—55.
	李琳璐. 教育研究范式的祛魅: 思辨与实证的融合共生大学教育科学 2021 03 31 38 李琳璐. (2021). 教育研究范式的祛魅: 思辨与实证的融合共生. 大学教育科学,(03),31—38.
	任思腾. 科学实验中的可重复概念自然辩证法通讯 2020 42 09 46 52 任思腾. (2020). 科学实验中的可重复概念. 自然辩证法通讯,42(09),46—52.
	沙沃森, 汤. (2006). 教育的科学研究(曹晓南等译). 北京: 教育科学出版社.
	王晓诚, 贾丽娜, 金元英. 家庭社会经济地位对初中生阅读成就的影响: 阅读动机和阅读活动的中介作用心理与行为研究 2020 06 839 845 王晓诚, 贾丽娜, 金元英. (2020). 家庭社会经济地位对初中生阅读成就的影响: 阅读动机和阅读活动的中介作用. 心理与行为研究,(06),839—845.
	阎光才. 教育及社会科学研究中的数据——兼议当前的大数据热潮北京大学教育评论 2013 11 04 77 86 阎光才. (2013). 教育及社会科学研究中的数据—兼议当前的大数据热潮. 北京大学教育评论,11(04),77—86. doi: 10.3969/j.issn.1671-9468.2013.04.005
	姚计海. 教育实证研究方法的范式问题与反思华东师范大学学报(教育科学版) 2017 03 64 71+169−170 姚计海. (2017). 教育实证研究方法的范式问题与反思. 华东师范大学学报(教育科学版),(03),64—71+169−170.
	袁振国. 实证研究是教育学走向科学的必要途径华东师范大学学报(教育科学版) 2017 35 03 4 17+168 袁振国. (2017). 实证研究是教育学走向科学的必要途径. 华东师范大学学报(教育科学版),35(03),4—17+168.
	张国平, 高耀明. 美国大学高等教育学专业研究生培养的特色及其启示高等教育研究 2019 9 53 60 张国平, 高耀明. (2019). 美国大学高等教育学专业研究生培养的特色及其启示. 高等教育研究,9,53—60.
	周明. 教育实证研究的地位与价值——中国教育学会教育理论刊物分会2015年年会综述教育科学 2016 02 93 96 周明. (2016). 教育实证研究的地位与价值—中国教育学会教育理论刊物分会2015年年会综述. 教育科学,(02),93—96. doi: 10.3969/j.issn.1002-8064.2016.02.015
	朱军文, 马银琦. 教育实证研究这五年: 特征、趋势及展望华东师范大学学报(教育科学版) 2020 09 16 35 朱军文, 马银琦. (2020). 教育实证研究这五年: 特征、趋势及展望. 华东师范大学学报(教育科学版),(09),16—35.
	Ajzen, I. The theory of planned behavior Organizational Behavior & Human Decision Processes 1991 50 2 179 211 Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50(2), 179—211.
	Bradley, R. H. & Corwyn, R. F. Socioeconomic status and child development Annual Review of Psychology 2002 53 371 399 Bradley, R. H. & Corwyn, R. F. (2002). Socioeconomic status and child development. Annual Review of Psychology, 53, 371—399. doi: 10.1146/annurev.psych.53.100901.135233
	Briggs, C. & Elkind, D. Characteristics of early readers Perceptual & Motor Skills 1977 44 3c 1231 1237 Briggs, C. & Elkind, D. (1977). Characteristics of early readers. Perceptual & Motor Skills, 44(3c), 1231—1237.
	Capraro, R. M. & Thompson, B. (2008). The educational researcher defined: what will future researchers be trained to do?. The Journal of Educational Research, 101(4), 247−253.
	Chen, C. P. Understanding mobile English-learning gaming adopters in the self-learning market: The uses and gratification expectancy model Computers & Education 2018 126 217 230 Chen, C. P. (2018). Understanding mobile English-learning gaming adopters in the self-learning market: The uses and gratification expectancy model. Computers & Education, 126, 217—230.
	Chhin, C. S., Taylor, K. A., & Wei, W. S. Supporting a culture of replication: An examination of education and special education research grants funded by the Institute of Education Sciences Educational Researcher 2018 47 9 594 605 Chhin, C. S., Taylor, K. A., & Wei, W. S. (2018). Supporting a culture of replication: An examination of education and special education research grants funded by the Institute of Education Sciences. Educational Researcher, 47(9), 594—605. doi: 10.3102/0013189X18788047
	Dear, P. (2001). Revolutionizing the sciences: European knowledge and its ambitions, 1500–1700. New York: Palgrave Macmillan.
	Durkin, D. (1966). Children who read early. New York: Teachers College Press.
	Enserink & Martin. Controversial studies give a deadly flu virus wings Science 2011 334 6060 1192 3 Enserink & Martin. (2011). Controversial studies give a deadly flu virus wings. Science, 334(6060), 1192—3. doi: 10.1126/science.334.6060.1192
	Fishbein, M., Ajzen, I. & Belief, A. Belief, attitude, intention, and behavior: An introduction to theory and research Contemporary Sociology 1977 6 2 244 245 Fishbein, M., Ajzen, I. & Belief, A. (1977). Belief, attitude, intention, and behavior: An introduction to theory and research. Contemporary Sociology, 6(2), 244—245. doi: 10.2307/2065853
	Francis, G. The psychology of replication and replication in psychology Perspectives on Psychological Science. A Journal of the Association for Psychological Science 2012 7 6 585 Francis, G. (2012). The psychology of replication and replication in psychology. Perspectives on Psychological Science. A Journal of the Association for Psychological Science, 7(6), 585. doi: 10.1177/1745691612459520
	Gelman, A. Don’t characterize replications as successes or failures The Behavioral and Brain Sciences 2018 41 128 20 Gelman, A. (2018). Don’t characterize replications as successes or failures. The Behavioral and Brain Sciences, 41, 128—20. doi: 10.1017/S0140525X18000638
	Gewin, V. Data sharing: An open mind on open data Nature 2016 529 7584 117 119 Gewin, V. (2016). Data sharing: An open mind on open data. Nature, 529(7584), 117—119. doi: 10.1038/nj7584-117a
	Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. Comment on “Estimating the reproducibility of psychological science” Science 2016 351 6277 1037 Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the reproducibility of psychological science”. Science, 351(6277), 1037.
	Harwell, M., Maeda, Y., Bishop, K. & Xie, A. The surprisingly modest relationship between SES and educational achievement The Journal of Experimental Education 2017 85 2 197 214 Harwell, M., Maeda, Y., Bishop, K. & Xie, A. (2017). The surprisingly modest relationship between SES and educational achievement. The Journal of Experimental Education, 85(2), 197—214. doi: 10.1080/00220973.2015.1123668
	Kier, M. W., Blanchard, M. R., Osborne, J. W. & Albert, J. L. The development of the STEM career interest survey (STEM-CIS) Research in Science Education 2014 44 3 461 481 Kier, M. W., Blanchard, M. R., Osborne, J. W. & Albert, J. L. (2014). The development of the STEM career interest survey (STEM-CIS). Research in Science Education, 44(3), 461—481. doi: 10.1007/s11165-013-9389-3
	Koyunlu Unlu, Z., Dokme, I., & Unlu, V. Adaptation of the science, technology, engineering, and mathematics career interest survey (STEM-CIS) into Turkish Eurasian Journal of Educational Research 2016 16 63 21 36 Koyunlu Unlu, Z., Dokme, I., & Unlu, V. (2016). Adaptation of the science, technology, engineering, and mathematics career interest survey (STEM-CIS) into Turkish. Eurasian Journal of Educational Research, 16(63), 21—36. doi: 10.14689/ejer.2016.63.2
	Levy, L. H. Reflections on replications and the experimenter bias effect Journal of Consulting and Clinical Psychology 1969 33 15 17 Levy, L. H. (1969). Reflections on replications and the experimenter bias effect. Journal of Consulting and Clinical Psychology, 33, 15—17. doi: 10.1037/h0027230
	Light, R. J. Measures of response agreement for qualitative data: some generalizations and alternatives Psychological Bulletin 1971 76 5 365 Light, R. J. (1971). Measures of response agreement for qualitative data: some generalizations and alternatives. Psychological Bulletin, 76(5), 365. doi: 10.1037/h0031643
	Madden, C. S., Easley, R. W. & Dunn, M. G. How Journal Editors View Replication Research Journal of Advertising 1995 24 4 77 87 Madden, C. S., Easley, R. W. & Dunn, M. G. (1995). How Journal Editors View Replication Research. Journal of Advertising, 24(4), 77—87. doi: 10.1080/00913367.1995.10673490
	Mahoney, M. J. Open exchange and epistemic progress American Psychologist 1985 40 1 29 39 Mahoney, M. J. (1985). Open exchange and epistemic progress. American Psychologist, 40(1), 29—39. doi: 10.1037/0003-066X.40.1.29
	Makel, M. C., Plucker, J. A. & Hegarty, B. Replications in psychology research: how often do they really occur? Perspectives on Psychological Science A Journal of the Association for Psychological Science 2012 7 6 537 Makel, M. C., Plucker, J. A. & Hegarty, B. (2012). Replications in psychology research: how often do they really occur?. Perspectives on Psychological Science A Journal of the Association for Psychological Science, 7(6), 537. doi: 10.1177/1745691612460688
	Makel, M. C. & Plucker, J. A. Facts are more important than novelty: Replication in the education sciences Educational Researcher 2014 43 6 304 316 Makel, M. C. & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304—316. doi: 10.3102/0013189X14545513
	Makel, M. C., Plucker, J. A., Freeman, J., Lombardi, A., Simonsen, B. & Coyne, M. Replication of special education research: Necessary but far too rare Remedial and Special Education 2016 31 2 48 53 Makel, M. C., Plucker, J. A., Freeman, J., Lombardi, A., Simonsen, B. & Coyne, M. (2016). Replication of special education research: Necessary but far too rare. Remedial and Special Education, 31(2), 48—53.
	Plucker, Jonathan. & Makel, Matthew. (2021). Replication is important for educational psychology: Recent developments and key issues. Educational Psychologist. 5(6), 1−11.
	Monya, B. 1, 500 scientists lift the lid on reproducibility Nature 2016 533 7604 452 454 Monya, B. (2016). 1, 500 scientists lift the lid on reproducibility. Nature, 533(7604), 452—454. doi: 10.1038/533452a
	Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Yarkoni, T. Promoting an open research culture Science 2015 348 6242 1422 1425 Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422—1425. doi: 10.1126/science.aab2374
	Ostrom, T. H. To replicate or explicate American Psychologist 1971 26 3 312 Ostrom, T. H. (1971). To replicate or explicate. American Psychologist, 26(3), 312.
	Pashler, H. & Harris, C. R. Is the replicability crisis overblown? Three arguments examined Perspectives in Psychological Science 2012 7 6 531 Pashler, H. & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives in Psychological Science, 7(6), 531. doi: 10.1177/1745691612463401
	Ritchie, S. J., Wiseman, R. & French, C. C. Replication, replication, replication Psychologist 2012 25 346 348 Ritchie, S. J., Wiseman, R. & French, C. C. (2012). Replication, replication, replication. Psychologist, 25, 346—348.
	Rosenthal, R. & Fode, K. L. The effect of experimenter bias on the performance of the albino rat Behavioral Science 1963 8 3 183 189 Rosenthal, R. & Fode, K. L. (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science, 8(3), 183—189.
	Rosenthal, R. & Jacobson, L. Pygmalion in the classroom Urban Review 1968 3 1 16 20 Rosenthal, R. & Jacobson, L. (1968). Pygmalion in the classroom. Urban Review, 3(1), 16—20. doi: 10.1007/BF02322211
	Schmidt, S. Shall we really do it again? The powerful concept of replication is neglected in the social sciences Review of General Psychology 2009 13 2 90 100 Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90—100. doi: 10.1037/a0015108
	Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant Psychological Science 2011 22 11 1359 1366 Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359—1366. doi: 10.1177/0956797611417632
	Sirin, S. R. Socioeconomic status and academic achievement: A meta-analytic review of research Review of Educational Research 2005 75 3 417 453 Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417—453. doi: 10.3102/00346543075003417
	Towne, L., Fletcher, J. M., Wise, L. L. & Winters, T. M. Advancing scientific research in education National Academies Press 2005 106 100 329 337 Towne, L., Fletcher, J. M., Wise, L. L. & Winters, T. M. (2005). Advancing scientific research in education. National Academies Press, 106(100), 329—337.
	Travers, J. C., Cook, B. G., Therrien, W. J. & MD Coyne. Replication research and special education Remedial & Special Education 2016 37 4 195 204 Travers, J. C., Cook, B. G., Therrien, W. J. & MD Coyne. (2016). Replication research and special education. Remedial & Special Education, 37(4), 195—204.
	Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. Contextual sensitivity in scientific reproducibility Proceedings of the National Academy of Sciences of the United States of America 2016 113 23 6454 6459 Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016). Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences of the United States of America, 113(23), 6454—6459. doi: 10.1073/pnas.1521897113
	Wentzel, K. R. Open science reforms: Strengths, challenges, and future directions Educational Psychologist 2021 56 2 161 173 Wentzel, K. R. (2021). Open science reforms: Strengths, challenges, and future directions. Educational Psychologist, 56(2), 161—173. doi: 10.1080/00461520.2021.1901709

类型	研究变量	对象总体	研究样本	研究方法
精确重复	不变	不变	不变	不变
直接重复	不变	不变	变	不变
			变	变
			不变	变
		变	变	不变
		变	变	变
概念重复	核心变量不变，非核心变量变	不变	不变	不变
			不变	变
			变	不变
			变	变
		变	变	不变
		变	变	变

	发文时间						总计
	2015	2016	2017	2018	2019	2020	总计
总载文量	6252	5945	6002	5702	5500	5327	34728
量化研究	532	521	472	519	503	472	3019
量化研究/总载文量	8.51%	8.76%	7.86%	9.10%	9.14%	8.86%	8.69%
重复性研究	17	20	19	21	17	31	125
重复性研究/量化研究	3.20%	3.84%	4.03%	4.05%	3.38%	6.57%	4.14%

[1]	Lyu Jing. The Quantitative Methods in Education Empirical Research in China: A Review on Five Years’ Application [J]. Journal of East China Normal University(Educational Sciences), 2020, 38(9): 36-55.
[2]	Du Yuhong, Zang Lin. Classification of Disciplines and Improved Quality of Quantitative Research of Education [J]. Journal of East China Normal University(Educationa, 2019, 37(4): 38-46.
[3]	CHENG Jiankun, CHEN Jing. Empirical Educational Research: Its Past, Present and Future [J]. Journal of East China Normal University(Educationa, 2017, 35(3): 150-158+174.