All About the Selection

A new paper by Josh Angrist and co-authors shows that top students that went to elite prep schools do no better than top students who did not go to elite prep schools.  In fact, the overall performance of elite prep schools is all down to the fact that they can select.  This is also my take on so-called "successful" charter schools - their success is often a result of selection: motivated families switch their kids while unmotivated ones don't.  Which is another lesson from the good research out there on schools: all the stuff that private schools can offer - nice facilities, homogeneous populations, so called 'rigorous' curriculum and so on, do not matter much in terms of actual academic achievement.  Their success is all about selection.

I refer to this thanks to Felix Salmon, whom I shall simply quote bacause I could not say it better myself:

There aren’t a lot of studies of public vs private schools, but the ones which do exist generally show no difference at all in educational outcome, once you control for the socio-economic status of the kids being admitted. Essentially: middle-class kids who grow up with two well-educated parents and lots of books around the house will generally do very well in school no matter where they go.

Which is why I am always try to be a calming voice to well-educated middle-class parents in PPS who are dismayed at the erosion of support for public education. It is generally not their kids that suffer.

Time in the Classroom Matters

As school districts across Oregon are preparing for drastic cuts including shortening the school year, a new study confirms, yet again, that instructional time matters for student achievement. Here is the abstract of the paper by Victor Lavy:

There are large differences across countries in instructional time in schooling institutions. Can these differences explain some of the differences across countries in pupils’ achievements in different subjects? While research in recent years provides convincing evidence about the effect of several inputs in the education production function, there is limited evidence on the effect of classroom instructional time. Such evidence is of policy relevance in many countries, and it became very concrete recently as President Barrack Obama announced the goal of extending the school week and year as a central objective in his proposed education reform for the US. In this paper, I estimate the effects of instructional time on students’ academic achievement in math, science and reading. I estimate linear and non-linear instructional time effects controlling for unobserved heterogeneity of both pupils and schools. The evidence from a sample of 15 year olds from over fifty countries that participated in PISA 2006 consistently shows that instructional time has a positive and significant effect on test scores. The effect is large relative to the standard deviation of the within pupil test score distribution. I obtain similar evidence from a sample of 10 and 13 year olds in Israel. The OLS results are highly biased upward but the within student estimates are very similar across groups of developed and middle-income countries and age groups. Evidence from primary and middle schools in Israel is similar to the evidence from OECD countries. However, the estimated effect of instructional time in the sample of developing countries is much lower than the effect size in the developed countries. I also show that the productivity of instructional time is higher in countries that implemented school accountability measures, and in countries that give schools autonomy in hiring and firing teachers.

Economist's Notebook: Schools, Big and Small

I am an interested party in the Portland high school redesign effort.  I will have children that I expect will attend Portland high schools.  In the case of my kids, they will most likely attend Cleveland, one of the high schools that seems to do reasonably well in various metrics.  The schools the district is worried about are schools like Jefferson, Franklin and Marshall that draw from lower socio-economic status areas and have worse performance metrics.  I understand the concern: you cannot sit idly by and let schools fail the students that depend on them most.

But I am very frustrated about what appears to be the following of the latest fads in educations causing new disruptions every few years.  Just a few years ago the miracle cure was small sized high schools that could provide needy students the personal attention they needed.  Of course, it sounds so obvious.  And yet, despite a mountain of educational research that suggested this was an established empirical fact, when the Gates Foundation spent lots of money promoting small schools and (to their great credit) carefully studying the result, they found that small schools were a complete failure.  Unfortunately, Portland was one of the great bandwagon districts that jumped on board wholeheartedly (and Vicki Phillips parlayed this experience into a job with the Gates Foundation).  Now, even PPS admits it was a failure - well, almost.  Here is their take:

Researchers have found that, other things being equal, the large size of many high schools is correlated with lower levels of student achievement, engagement and graduation rates, particularly among poor and minority students.

However, the small school reform movement that followed this research in such cities as Portland, Boston, New York, San Diego and Chicago, has shown mixed results. The Gates Foundation’s 2009 Annual Letter — after the foundation invested millions in converting large, comprehensive high schools into small schools — said those small schools did not improve students’ achievement in any significant way. Those that did were almost all charter schools that had significantly longer school days. The foundation noted that it had less success trying to change an existing school than helping to create a new school.

Jefferson, Madison, Marshall and Roosevelt, the Portland high schools that moved toward a small school model, whether into fully autonomous small schools (at Marshall and Roosevelt next year) or trying to create smaller “academies” within a school — were all attempts to convert a large, existing school with poor performance into more successful small schools. However, those campuses are still posting some of the higher dropout and absentee rates in the district, and many still are showing some of the lowest scores on Oregon assessment tests. In addition, community members say students do not have access to essential courses and electives.

However, the growth data tells a different story: Most of the schools are making significant progress is getting more students to benchmark, and individual students with low scores in eighth grade are making great gains by the 10th-grade assessments, even if they still might not meet benchmark. Our small schools have demonstrated the highest academic growth rates in both reading and math for four straight years. The Spanish-English International School on the Roosevelt Campus doubled its graduation rate between 2004 and 2008, and last year bested the district average. This school also has claimed the highest academic growth rate on state assessments in reading for three straight years.

So amusing as this is: 'research says small size is best, but experience says it is not, including Portland, except they seem to be getting better,' it is neither particularly helpful nor a particularly useful roadmap for where to go next. Yet this does not seem to deter PPS.

But what I have discovered is just how bad the education research is.  Some of the most basic themes in social science empirical research - correlation vs. causation, selection, endogeneity - appear to have escaped the attention of most researchers.  The common type of school size study I have found is the type that looks at small and large schools and compares them.  I would hope that most of my students could immediately spot about ten things wrong with drawing inference from such a study.  So it is no wonder that when the Gates Foundation studied the effects of downsizing schools it was dissappionted. 

There may be many things to complain about with Arne Duncan's new (or old depending on your perspective) education policies but the one I am extremely pleased about is the focus on the collection and careful study of real world data so that we can finally have some good answers to some basic education questions.  We should have evidence-based policy but to do so we need good evidence that is well understood.