Despite the Coleman Report half a century ago, there is no consensus on which is more important, parenting or schooling, for students’ academic performance. Based on the large-scale assessment data from primary and secondary schools in 16 cities of 5 provinces in eastern and central China, this study used generalized educational production function and established the hierarchical linear model (HLM) to analyze the impact of parenting and schooling on students’ standardized test scores. In addition, we employed Shapley and Owen decomposition method to identify the input factors that have a greater impact on school output and drew the following conclusions. Firstly, in addition to the student-teacher ratio, school input factors such as school conditions and teacher quality have significantly positive effects on students’ academic performance. Secondly, parental participation and parental educational expectations have significantly positive effects on students’ academic performance. Thirdly, parenting factors are more important for primary school students while schooling factors are more important for secondary ones. Finally, compared with school conditions, teacher quality contributes more to the variance of average academic performance in primary and secondary schools. Moreover, compared with primary schools, teacher quality has greater contribution to the variance of average academic performance in secondary schools. Based on the empirical research conclusions, we put forward five suggestions to improve the productivity of primary and secondary education in China. Firstly, the government should adjust the allocation of resources for compulsory education and give priority to funding for secondary school education. Secondly, the government should raise teachers’ salary and improve the work environment to attract high-quality labor into compulsory education. Thirdly, the government and schools should enhance teachers’ teaching through teacher training and cooperation. Fourthly, the government should make rules and regulations about family education to enhance parents’ responsibility in their children’s education. Finally, schools and communities should organize some lectures on family education to promote effective cooperating between schooling and parenting.