美国基础教育项目效果评估经验何在？——基于25项高质量评估研究的主题文本分析

doi:10.16382/j.cnki.1000-5560.2022.01.004

摘要/Abstract

摘要：

美国基础教育项目效果评估不仅实践起步早、根基相对坚实，而且目前也已发展成为教育科学研究的重要类型以及循证教育改革的证据来源，积累了较为成熟的实践经验。本研究对25项高质量评估研究进行主题文本分析后发现，其实践经验主要表现为：第一，充足的评估经费是评估的首要前提。评估经费来源于以政府为首的各界支持，经费筹措是集体同心的智慧行动，经费使用贯穿评估全程。第二，专业的评估人员是评估的核心输入。评估人员来自高校等专业组织，其选择确定往往会历经多方因素的全面考量，目前存在内部、外部、内外合作等三种评估人员角色定位。第三，科学的评估开展是评估的关键过程。评估规划会确定评估类型与问题，评估设计会厘清理论基础、评估方法和效果指标，评估实施会做好样本招募、干预实施以及数据收集。第四，实用的评估成果是评估的重要输出。评估发现会被恰当分析与合理解释，评估结果会被规范撰写与多样呈现，评估产品还会被进一步大力传播与有效使用。此外，美国基础教育项目效果评估在努力提升质量的同时也开始不断迸发出新的实践趋势。

关键词: 美国基础教育, 《每个学生都成功法》, 循证教育改革, 项目效果评估, 教育实验研究, 主题文本分析

Abstract:

The impact evaluation of PK-12 educational programs in the United States not only started early in practice, with a relatively solid foundation, but also has developed into an important type of educational scientific research and the evidence source for the evidence-based educational reform at present, accumulating a lot of relatively mature practical experience. After thematically analyzing the text of 25 high-quality evaluation research, this research found that its practical experience is as follows. First of all, adequate evaluation funding is the first prerequisite for evaluation. The funding support comes from government and other sources, the raising is a collective action of wisdom, and the allocation runs through the entire evaluation process. Secondly, professional evaluators are the core input of evaluation. The evaluators come from professional organizations such as universities and colleges, and appropriate evaluators will be carefully selected and determined based on many factors, and there are currently three types of evaluator role positioning, namely, internal evaluators, external evaluators, and evaluators who collaborate internally and externally. Thirdly, scientific evaluation conducting is the key process of evaluation. The evaluation planning usually determines the evaluation type and questions. The evaluation design will clarify the theoretical basis, evaluation methods and outcome measures. And the evaluation implementation will attach great importance to sample recruitment, intervention implementation and data collecting. Fourthly, utility fruit is an important output of evaluation. The evaluation findings will be properly analyzed and reasonably explained. The evaluation results will be written by specifications and presented in various ways. And the evaluation products will be vigorously disseminated and used effectively. In addition, while striving to improve its quality, the impact evaluation of PK-12 educational programs in the U.S. has also begun to burst out new practical trends.

Key words: PK-12 education in the U.S., Every Student Succeeds Act, evidence-based reform in education, program impact evaluation, educational experiment research, thematic text analysis

时晨晨. 美国基础教育项目效果评估经验何在？——基于25项高质量评估研究的主题文本分析[J]. 华东师范大学学报(教育科学版), 2022, 40(1): 43-59.

Chenchen Shi. What is the Practical Experience of Impact Evaluation of PK-12 Educational Programs in the United States:Thematic Text Analysis Based on 25 High-Quality Evaluation Research[J]. Journal of East China Normal University(Educational Sciences), 2022, 40(1): 43-59.

图/表 1

表 1

25项高质量学生出勤改进项目效果评估研究"

学生出勤改进项目中英名称	高质量效果评估研究 APA著录	高质量效果评估研究中文标题
出勤和旷课干预及普遍程序（Attendance and Truancy Intervention and Universal Procedures）	Berg，2018	《我们能利用一个普遍预防项目来增加出勤和减少慢性缺勤吗？一项对“出勤和旷课干预及普遍程序”项目的随机对照实验》
促进高中毕业与高等教育的长期改善（Facilitating Long-term Improvements in Graduation and Higher Education for Tomorrow）	Philp，2015	《“促进高中毕业与高等教育的长期改善”项目：最终评估报告》
大哥哥大姐姐（Big Sisters Big Brothers）	Herrera et al.，2007	《在学校中有所作为：“大哥哥大姐姐”项目校本指导干预的影响研究》
第二步（Second Step）	Neace & Mu?oz，2012	《推倒教育的界限：“第二步”项目效果评估——一门具有社会心理和非认知干预的暴力预防课程》
积极家庭支持（Positive Family Supports）	Smolkowski et al.，2017	《“积极家庭支持”项目的效能评估：一个面向初中学校的三层级公共卫生传递模型》
检查与连接（Check & Connect）	Guryan et al.，2017	《指导干预对学校出勤和学业成就的影响：一项对“检查与连接”项目的随机评估》
发短信（text messaging）	Balu，Porter，& Gunton，2016	《给家长发短信能够帮助改进高中生的学校出勤吗？》
放学后教育重要（Afterschool Matters）	Hirsch et al.，2011	《面向高中生的放学后教育项目：一项对“放学后教育重要”项目的评估研究》
星火（SPARK）	Jones，Christian，& Rice，2016	《“星火”早期读写项目的随机对照实验评估结果》
学校早餐项目试点计划（School Breakfast Program Pilot Project）	Bernstein et al.，2004	《“学校早餐项目试点计划”评估：最终报告》
艺术通道倡议（Art Access Initiative）	Bowen & Kisida，2019	《艺术学习体验的因果效应研究：来自休斯顿“艺术倡议通道”项目的实验证据》
阅读、写作、尊重和解决（Reading，Writing，Respect and Resolution）	Jones，Brown，& Lawrence，2011	《面向全体学生的社会—情感与读写校本干预的实施效果：一项转化发展研究的两年实验》
早期预警干预和监督系统（Early Warning Intervention and Monitoring System）	Faria et al.，2017	《使学生走上毕业的正轨：“早期预警干预和监督系统”项目实施一年后的效果》
追求公平和恢复性社区（Pursuing Equitable and Restorative Communities）	Augustine et al.，2018	《恢复性实践能够改进学校氛围和减少停课停学吗？一项对恢复性实践在一个中等城市学区里实施效果的评估》
成为一名男子汉（Becoming A Man）	Heller et al.，2013	《青少年暴力与辍学预防：一项随机田野实验》
成为一名男子汉（Becoming A Man）	Heller et al.，2017	《对三项在芝加哥开展的减少犯罪与辍学实验的思考》
积极行为（Positive Action）	Snyder et al.，2010	《社会—情感和品格发展项目对学校层面上学业成就、缺勤和纪律结果的影响：一项随机对照实验》
积极行为（Positive Action）	Bavarian et al.，2013	《利用社会—情感和品格发展来改进学业成就：一项在低收入城区学校内开展的随机对照实验》
每日在校实验室（EveryDay Labs）	Robinson et al.，2018	《通过干预家长观念来减少低年级学生的缺勤》
每日在校实验室（EveryDay Labs）	Rogers & Feller，2018	《通过干预家长错误观念来大规模减少学生缺勤》
学校中的社区（Communities In Schools）	Figlio，2015	《“学校中的社区”芝加哥伙伴项目对学生学业成就影响的实验证据》
	ICF International，2010a	《“学校中的社区”项目全国评估第4卷：随机对照实验研究（杰克逊维尔，佛罗里达州）》
	ICF International，2010b	《“学校中的社区”项目全国评估第5卷：随机对照实验研究（奥斯汀，德克萨斯州）》
	ICF International，2010c	《“学校中的社区”项目全国评估第6卷：随机对照实验研究（威奇托，堪萨斯州）》
	Parise et al.，2017	《“学校中的社区”项目案例管理干预：两年随机对照实验评估的最终发现》

表 1

参考文献 0

	时晨晨. (2020). 实验与实践的融合: 美国循证教育改革研究(博士学位论文). 北京: 北京师范大学.
	吴康宁教育改革成功的基础教育研究 2012 33 1 24 31 吴康宁. (2012). 教育改革成功的基础. 教育研究,33(1),24—31.
	Angrist, J. D American education research changes tack Oxford Review of Economic Policy 2004 20 2 198 212 Angrist, J. D. (2004). American education research changes tack. Oxford Review of Economic Policy, 20(2), 198—212. doi: 10.1093/oxrep/grh011
	Augustine, C. H., Engerg, J., Grimm, G. E., et al. (2018). Can restorative practices improve school climate and curb suspensions? An evaluation of the impact of restorative practices in a mid-sized urban school district. Santa Monica, CA: RAND Corporation.
	Balu, R., Porter, K., & Gunton, B. (2016). Can informing parents help high school students show up for school. New York: Manpower Demonstration Research Corporation.
	Bavarian, N., Lewis, K. M., Dubois, D. L., et al Using social-emotional and character development to improve academic outcomes: A matched-pair, cluster-randomized controlled trial in low-income, urban schools Journal of School Health 2013 83 11 771 779 Bavarian, N., Lewis, K. M., Dubois, D. L., et al. (2013). Using social-emotional and character development to improve academic outcomes: A matched-pair, cluster-randomized controlled trial in low-income, urban schools. Journal of School Health, 83(11), 771—779. doi: 10.1111/josh.12093
	Berg, T. (2018). Can we increase attendance and decrease chronic absenteeism with a universal prevention program? A randomized control study of Attendance and Truancy Universal Procedures and Interventions(Dissertation). Eugene, OR: University of Oregon.
	Bernstein, L. S., McLaughlin, J. E., Crepinsek, M. K., et al. (2004). Evaluation of the School Breakfast Program Pilot Project: Final report. Alexandria, VA: U. S. Department of Agriculture.
	Borman, G. D Experiments for educational evaluation and improvement Peabody Journal of Education 2002 77 4 7 27 Borman, G. D. (2002). Experiments for educational evaluation and improvement. Peabody Journal of Education, 77(4), 7—27. doi: 10.1207/S15327930PJE7704_2
	Bowen, D. H., & Kisida, A. B. (2019). Investigating causal effects of arts education experiences: Experimental evidence from Houston’s Arts Access Initiative. Houston, TX: Kinder Institute for Urban Research, Rice University.
	Earle, J., Maynard, R., Neild, R. C., et al. (2013). Common guidelines for education research and development. Washington, DC: Institute of Education Sciences and National Science Foundation.
	Faria, A. M., Sorensen, N., Heppen, J., et al. (2017). Getting Students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year(REL 2017-272). Washington, DC: Regional Educational Laboratory Midwest.
	Figlio, D. (2015). Experimental evidence of the effects of the Communities In Schools of Chicago partnership program on student achievement. Evanston, IL: Northwestern University.
	Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines(4th ed.). Boston, MA: Pearson, 2011.
	Flay, B. R., Biglan, A., Boruch, R. F., et al Standards of evidence: Criteria for efficacy, effectiveness and dissemination Prevention Science 2005 6 3 151 175 Flay, B. R., Biglan, A., Boruch, R. F., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151—175. doi: 10.1007/s11121-005-5553-y
	Giancola, S. P. (2014). Evaluation matters: Getting the information you need from your evaluation. Kennett Square, PA: Giancola Research Associates, Inc.
	Gottfredson, D. C., Cook, T. D., Gardner, F. E., et al Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next Generation Prevention Science 2015 16 7 893 926 Gottfredson, D. C., Cook, T. D., Gardner, F. E., et al. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next Generation. Prevention Science, 16(7), 893—926. doi: 10.1007/s11121-015-0555-x
	Guryan, J., Christenson, S., Claessens, A., et al. (2017). The effect of mentoring on school attendance and academic outcomes: A randomized evaluation of the Check & Connect program. Evanston, IL: Northwestern University.
	Hatry, H. P., Winnie, R. E., & Fisk, D. M. (1973). Practical program evaluation for state and local government officials. Washington, DC: The Urban Institute.
	Heller, S., Pollack, H. A., Ander, R., et al. (2013). Preventing youth violence and dropout: A randomized field experiment. Cambridge, MA: National Bureau of Economic Research.
	Heller, S., Shan, A. K., Guryan, J., et al Thinking, fast and slow? Some field experiments to reduce crime and dropout in Chicago The Quarterly Journal of Economics 2017 132 1 1 54 Heller, S., Shan, A. K., Guryan, J., et al. (2017). Thinking, fast and slow? Some field experiments to reduce crime and dropout in Chicago. The Quarterly Journal of Economics, 132(1), 1—54. doi: 10.1093/qje/qjw033
	Herrera, C., Grossman, J. B., Kauh, T. J., et al. (2007). Making a difference in schools: The Big Brothers Big Sisters school-based mentoring impact study. Philadelphia, PA: Public/Private Ventures.
	Hirsch, B. J., Hedges, L. V., Stawicki, J., et al. (2011). After-school programs for high school students: An evaluation of After School Matters. Evanston, IL: Northwestern University.
	ICF International. (2010a). Communities In Schools national evaluation volume 4: Randomized controlled trial study(Jacksonville, Florida). Fairfax, VA: ICF International.
	ICF International. (2010b). Communities In Schools national evaluation volume 5: Randomized controlled trial study(Austin, Texas). Fairfax, VA: ICF International.
	ICF International. (2010c). Communities In Schools national evaluation volume 6: Randomized controlled trial study(Wichita, Kansas). Fairfax, VA: ICF International.
	Jones, C. J., Christian, M., & Rice, A. (2016). The results of a randomized control trial evaluation of the SPARK literacy program. Washington, DC: Society for Research on Educational Effectiveness.
	Jones, S. M., Brown, J. L., & Lawrence, A. J Two-year impacts of a universal school-based social-emotional and literacy intervention: An experiment in translational developmental research Child Development 2011 82 2 533 554 Jones, S. M., Brown, J. L., & Lawrence, A. J. (2011). Two-year impacts of a universal school-based social-emotional and literacy intervention: An experiment in translational developmental research. Child Development, 82(2), 533—554. doi: 10.1111/j.1467-8624.2010.01560.x
	Kuckartz, U. (2014). Qualitative text analysis: A guide to methods, practice and using software. Thousand Oaks, CA: Sage Publications.
	Mertens, D. M., & Wilson, A. T. (2019). Program evaluation theory and practice: A comprehensive guide(2nd ed.). New York: The Guilford Press.
	Neace, W. P., & Muñoz, M. A Pushing the boundaries of education: Evaluating the impact of Second Step®: A violence prevention curriculum with psychosocial and non-cognitive measures Child & Youth Services 2012 33 1 46 69 Neace, W. P., & Muñoz, M. A. (2012). Pushing the boundaries of education: Evaluating the impact of Second Step®: A violence prevention curriculum with psychosocial and non-cognitive measures. Child & Youth Services, 33(1), 46—69.
	Parise, L. M., Corrin, W., Granito, K., et al. (2017). Two years of case management: Final findings from the Communities In Schools random assignment evaluation. New York: Manpower Demonstration Research Corporation.
	Philp, J. D. (2015). FLIGHT: Final evaluation report. Columbia, SC: The Evaluation Group.
	Robinson, C. D., Lee, M. G., Dearing, E., et al Reducing student absenteeism in the early grades by targeting parental beliefs American Educational Research Journal 2018 55 6 1163 1192 Robinson, C. D., Lee, M. G., Dearing, E., et al. (2018). Reducing student absenteeism in the early grades by targeting parental beliefs. American Educational Research Journal, 55(6), 1163—1192. doi: 10.3102/0002831218772274
	Rogers, T., & Feller, A Reducing student absences at scale by targeting parents’ misbeliefs Nature Human Behaviour 2018 2 5 335 342 Rogers, T., & Feller, A. (2018). Reducing student absences at scale by targeting parents’ misbeliefs. Nature Human Behaviour, 2(5), 335—342. doi: 10.1038/s41562-018-0328-1
	Savage, J. D. (2000). Funding science in America: Congress, universities, and the politics of the academic pork barrel. Cambridge, UK: Cambridge University Press.
	Shi, C., Inns, A., Lake, C., & Slavin, R. E. (2019). Effective school-based programs for K-12 students’ attendance: A best-evidence synthesis. Baltimore, MD: Center for Research and Reform in Education, Johns Hopkins University.
	Slavin, R. E. (2018, November 29). A warm welcome from Babe Ruth’s home town to the Registry of Efficacy and Effectiveness Studies (REES)[Blog post]. Retrieved from https://robertslavinsblog.wordpress.com/2018/11/29/a-warm-welcome-from-babe-ruths-home-town-to-the-registry-of-efficacy-and-effectiveness-studies-rees/.
	Slavin R. E. (2019, October 24). Developer- and researcher-made measures[Blog post]. Retrieved from https://robertslavinsblog.wordpress.com/2019/10/24/developer-and-researcher-made-measures/.
	Slavin R. E How evidence-based reform will transform research and practice in education Educational Psychologist 2020 55 1 21 31 Slavin R. E. (2020). How evidence-based reform will transform research and practice in education. Educational Psychologist, 55(1), 21—31. doi: 10.1080/00461520.2019.1611432
	Slavin, R. E., & Cheung, A Lessons learned from large-scale randomized experiments Journal of Education for Students Placed at Risk 2017 22 4 253 259 Slavin, R. E., & Cheung, A. (2017). Lessons learned from large-scale randomized experiments. Journal of Education for Students Placed at Risk, 22(4), 253—259. doi: 10.1080/10824669.2017.1360774
	Smolkowski, K., Seeley, J. R., Gau, J. M., et al Effectiveness evaluation of the Positive Family Support intervention: A three-tiered public health delivery model for middle schools Journal of School Psychology 2017 62 6 103 125 Smolkowski, K., Seeley, J. R., Gau, J. M., et al. (2017). Effectiveness evaluation of the Positive Family Support intervention: A three-tiered public health delivery model for middle schools. Journal of School Psychology, 62(6), 103—125.
	Snyder, F., Flay, B., Vuchinich, S., et al Impact of a social-emotional and character development program on school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, cluster-randomized, controlled trial Journal of Research on Educational Effectiveness 2010 3 1 26 55 Snyder, F., Flay, B., Vuchinich, S., et al. (2010). Impact of a social-emotional and character development program on school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, cluster-randomized, controlled trial. Journal of Research on Educational Effectiveness, 3(1), 26—55.
	U. S. Institute of Education Sciences. (2020). Education research grants request for applications beginning in fiscal year 2021. Washington, DC: Institute of Education Sciences.
	Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies(2nd ed.). Upper Saddle River, NJ: Prentice Hall.