Cost-effectiveness Analysis in Randomized Controlled Trials: Methods and Practice in China’s Rural Education

doi:10.16382/j.cnki.1000-5560.2020.08.002

Abstract

Abstract: In recent years, scholars in the field of education have carried out a large number of randomized controlled trials (RCTs) and provided many evidence-based interventions to improve the academic performance and to increase enrollment rate from different perspectives. However, given limited resources, how can researchers compare among different options of interventions and choose the most effective plan? Such a question can be answered by conducting cost-effectiveness analysis. This paper hereby provides a detailed introduction to basic methods of cost-effectiveness analysis, including cost accounting methods and the interpretation of effectiveness. It also introduces a case to illustrate the steps of conducting a cost-effectiveness analysis. Further, the cost-effectiveness of RCTs in education conducted worldwide are summarized, which provide valuable implications for policy-makers to promote rural education in China. In conclusion, researchers suggest the cost-effectiveness of interventions should be included in the analysis of RCT results, so as to facilitate the comparisons among different projects and to provide more powerful evidence for policy-makers.

Key words: cost-effectiveness analysis, randomized controlled trials, social cost, marginal cost, Monte Carlo Simulations

Nie Jingchun, Gao Qiufeng, Yang Jie, Guan Hongyu, Cai Jianhua. Cost-effectiveness Analysis in Randomized Controlled Trials: Methods and Practice in China’s Rural Education[J]. Journal of East China Normal University(Educational Sciences), 2020, 38(8): 68-91.

References

Abeberese, A. B., Kumler, T. J., Linden, L. L. (2014). Improving reading skills by encouraging children to read: A randomized evaluation of the Sa Aklat Sisikat Reading Program in the Philippines. Journal of Human Resources, 49(3), 611?633
Amy, D., Paul, G., Suzanne, W., Bixuan, S. (2019). What education policies and programmes affect learning and time in school in developing countries? A review of evaluations from 1990 to 2014. Review of Education, 7(2), 295?387
Anand, S., Hanson, K. (1997). Disability-adjusted life years: A critical review. Journal of Health Economics, 16(6), 685?702
Auriol, E., Warlters, M. (2012). The marginal cost of public funds and tax reform in Africa. Journal of Development Economics, 97(1), 58?72
Bai, Y., Tang, B., Wang, B., Di Mo, Zhang, L., Rozelle, S., et al. (2018). Impact of online computer assisted learning on education: Evidence from a randomized controlled trial in China. Working paper.
Baird, S., McIntosh, C., Ozler, B. (2011). Cash or condition? Evidence from a cash transfer experiment. Quarterly Journal of Economics, 126(4), 1709?1753
Baltussen, R., Taghreed, A., Torres, T. T., Hutubessy, R., Murray, C. J. (2004). Making choices in health: Who guide to cost-effectiveness analysis. Geneva: World Health Organization.
Banerjee, A. V., Cole, S., Duflo, E., Linden, L. (2007). Remedying education: Evidence from two randomized experiments in India. Quarterly Journal of Economics, 122(3), 1235?1264
Barreraosorio, F., Linden, L. (2009). The use and misuse of computers in education: Evidence from a randomized controlled trial of a language arts program. Working paper.
Blimpo, M., Evans, D. (2011). School-based management and educational outcomes: Lessons from a randomized field experiment. Working paper.
Borkum, E., Fang, H., Linden, L. L. (2009). School libraries and language skills in Indian primary schools: A randomized evaluation of the Akshara Library Program. Working paper.
Burde, D., Linden, L. L. (2012). The effect of village-based schools: Evidence from a randomized controlled trial in Afghanistan. Iza Discussion Papers, 5.
Cristia, J., Ibarraran, P., Cueto, S., Santiago, A., Severin, E. (2012). Technology and child development: Evidence from the One Laptop per Child Program. Social Science Electronic Publishing, 4(4), 206?207
Dhaliwal, I., Duflo, E., Glennerster, R., Tulloch, C. (2013). Comparative cost-effectiveness analysis to inform policy in developing countries: A general framework with applications for education. In Education policy in developing countries, 285?338. Chicago: University of Chicago Press.
Drummond, M., Manca, A., Sculpher, M. (2005). Increasing the generalizability of economic evaluations: Recommendations for the design, analysis, and reporting of studies. International Journal of Technology Assessment in Health Care, 21(02), 165?171
Duflo, E., Dupas, P., Kremer, M. (2011). Peer effects, teacher incentives, and the impact of tracking: Evidence from a randomized evaluation in Kenya. American Economic Review, 101(5), 1739?1774
Duflo, E., Dupas, P., Kremer, M. (2015). School governance, teacher incentives, and pupil-teacher ratios: Experimental evidence from Kenyan primary schools. Journal of Public Economics, 123, 92?110
Duflo, E., Glennerster, R., Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of Development Economics, 4, 3895?3962
Duflo, E., Hanna, R., Ryan, S. (2012). Incentives work: Getting teachers to come to school. American Economic Review, 102(4), 1241?1278
Evans, D. K., Popova, A. (2016). Cost-effectiveness analysis in development: Accounting for local costs and noisy impacts. World Development, 77, 262?276
Glennerster, R., Takavarasha, K. (2013). Running randomized evaluations: A practical guide. Princeton: Princeton University Press.
Glewwe, P., Ilias, N., Kremer, M. (2010). Teacher incentives. American Economic Journal: Applied Economics, 2(3), 205?227
Glewwe, P., Kremer, M., Moulin, S. (2009). Many children left behind? Textbooks and test scores in Kenya. American Economic Journal: Applied Economics, 1(1), 112?135
Glewwe, P., Kremer, M., Moulin, S., Zitzewitz, E. (2004). Retrospective vs. prospective analyses of school inputs: The case of flip charts in Kenya. Journal of Development Economics, 74(1), 251?268
Glewwe, P., Park, A., Zhao, M. (2016). A better vision for development: Eyeglasses and academic performance in rural primary schools in China. Journal of Development Economics, 122, 170?182
Kleiman-Weiner, M., Luo, R., Zhang, L., Shi, Y., Medina, A., Rozelle, S. (2013). Eggs versus chewable vitamins: Which intervention can increase nutrition and test scores in rural China?. China Economic Review, 24, 165?176
Kremer, M., Miguel, E., Thornton, R. (2009). Incentives to learn. Review of Economics & Statistics, 91(3), 437?456
Lai, F., Luo, R., Zhang, L., Huang, X., Rozelle, S. (2015). Does computer-assisted learning improve learning outcomes? Evidence from a randomized experiment in migrant schools in Beijing. Economics of Education Review, 47, 34?48
Lai, F., Zhang, L., Bai, Y., Liu, C., Shi, Y., Chang, F., et al. (2016). More is not always better: Evidence from a randomised experiment of computer-assisted learning in rural minority schools in Qinghai. Journal of Development Effectiveness, 8(4), 449?472
Lai, F., Zhang, L., Hu, X., Qu, Q., Shi, Y., Qiao, Y., et al. (2013). Computer assisted learning as extracurricular tutor? Evidence from a randomised experiment in rural boarding schools in Shaanxi. Journal of Development Effectiveness, 5(2), 208?231
Levin, H. M., McEwan, P. J. (2001). Cost-effectiveness analysis: Methods and applications. Thousand Oaks: Sage.
Li, F., Song, Y., Yi, H., Wei, J., Zhang, L., Shi, Y., et al. (2016). The impact of conditional cash transfers on the matriculation of junior high school students into rural China's high schools. Journal of Development Effectiveness, 9(1), 41?60
Luo, R., Miller, G., Rozelle, S., Sylvia, S., Vera-Hernández, M. (2020). Can bureaucrats really be paid like ceos? Substitution between incentives and resources among school administrators in China. Journal of the European Economic Association, 18(1), 165?201
Luo, R., Shi, Y., Zhang, L., Liu, C., Rozelle, S., Sharbono, B., et al. (2012). Nutrition and educational performance in rural China's elementary schools: Results of a randomized control trial in Shaanxi province. Economic Development and Cultural Change, 60(4), 735?772
Ma, X., Zhou, Z., Yi, H., Pang, X., Shi, Y., Chen, Q., et al. (2014). Effect of providing free glasses on children's educational outcomes in China: Cluster randomized controlled trial. BMJ, 349, 1?12
McEwan, P. J. (2012). Cost-effectiveness analysis of education and health interventions in developing countries. Journal of Development Effectiveness, 4(2), 189?213
Miller, G., Luo, R., Zhang, L., Sylvia, S., Shi, Y., Foo, P., et al. (2012). Effectiveness of provider incentives for anaemia reduction in rural China: A cluster randomised trial. BMJ, 345, 1?10
Mo, D., Huang, W., Shi, Y., Zhang, L., Boswell, M., Rozelle, S. (2015). Computer technology in education: Evidence from a pooled study of computer assisted learning programs among rural students in China. China Economic Review, 36, 131?145
Mo, D., Swinnen, J., Zhang, L., Yi, H., Qu, Q., Boswell, M., et al. (2013). Can one-to-one computing narrow the digital divide and the educational gap in China? The case of Beijing migrant schools. World Development, 46, 14?29
Mo, D., Zhang, L., Luo, R., Qu, Q., Huang, W., Wang, J., et al. (2014). Integrating computer-assisted learning into a regular curriculum: Evidence from a randomised experiment in rural schools in Shaanxi. Journal of Development Effectiveness, 6(3), 300?323
Mo, D., Zhang, L., Yi, H., Luo, R., Rozelle, S., Brinton, C. (2013). School dropouts and conditional cash transfers: Evidence from a randomised controlled trial in rural China's junior high schools. Journal of Development Studies, 49(2), 190?207
Muralidharan, K., Sundararaman, V. (2010). The impact of diagnostic feedback to teachers on student learning: Experimental evidence from India. The Economic Journal, 120(546), F187?F203
Nguyen, T. (2008). Information, role models and perceived returns to education: Experimental evidence from Madagascar. Working paper.
Nie, J., Pang, X., Wang, L., Rozelle, S., Sylvia, S. (2020). Seeing is believing: Experimental evidence on the impact of eyeglasses on academic performance, aspirations, and dropout among junior high school students in rural China. Economic Development and Cultural Change, 68(2), 335?355
Pradhan, M., Suryadarma, D., Beatty, A., Wong, M., Gaduh, A., Alisjahbana, A., et al. (2014). Improving educational quality through enhancing community participation: Results from a randomized field experiment in Indonesia. American Economic Journal: Applied Economics, 6(2), 105?126
Siegel, J. E., Weinstein, M. C., Russell, L. B., Gold, M. R. (1996). Recommendations for reporting cost-effectiveness analyses. JAMA, 276(16), 1339?1341
Wong, H. L., Shi, Y., Luo, R., Zhang, L., Rozelle, S. (2013). Improving the health and education of elementary schoolchildren in rural China: Iron supplementation versus nutritional training for parents. Journal of Development Studies, 50(4), 502?519
Yi, H., Song, Y., Liu, C., Huang, X., Zhang, L., Bai, Y., et al. (2015). Giving kids a head start: The impact and mechanisms of early commitment of financial aid on poor students in rural China. Journal of Development Economics, 113, 1?15
Zhang, L., Cai, Y., Wang, X., Ma, X., Medina, A., Smith, D. S., et al. (2013). Neglected Rural Public Health Issue: The Case of Intestinal Roundworms. China & World Economy, 3, 25?43.
Zhuang, J., Liang, Z., Lin, T., Guzman, F. D. (2007). Theory and practice in the choice of social discount rate for cost-benefit analysis: A survey. Working paper.

[1]	Shi Yaojiang, Zhang Linxiu, Chang Fang, Liu Han, Tang Bin, Gao Qiufeng, Guan Hongyu, Nie Jingchun, Yang Jie, Bai Yu, Li Ying, Tang Lei, Yue Ai, Ru Tong. Practices and Experiences of Randomized Controlled Trials in China’s Targeted Poverty Alleviations in Education [J]. Journal of East China Normal University(Educational Sciences), 2020, 38(8): 1-67.
[2]	Shi Rong, Shi Yaojiang, Bill Bikales, Ricardo Morel. Randomized Controlled Trials and Its Policy Implication on Poverty Relief Globally [J]. Journal of East China Normal University(Educational Sciences), 2020, 38(8): 110-125.