Thursday, February 24, 2011

Peer Effects in Education

Steve Duin has a nice column today in the O about how little the PPS high school redesign does to improve the socio-economic heterogeneity of Portland's public high schools.

He read along -- in "The Big Ideas Behind the Big Ideas" -- as PPS lauded the "groundbreaking" 1966 study that decreed that after the poverty level at a child's home, "the biggest predicator of academic achievement is the socioeconomic status of the school he or she attends (even more so than per pupil spending)."

Then Rupp tripped over page 30 and the district's own projections on how dramatically the redesign will impact the free and reduced-price lunch rates at its comprehensive high schools the next 10 years.

By and large, it won't.

My first thought was "1966, really? You have got to be kidding me." The ability of social science researchers to deal with data in 1966 was almost nil, and the ability to isolate something as tricky as peer effects was precisely nil. That PPS would still use such an outdated study worries me, it worries me a lot. Especially since there have been very very good studies of peer effects conducted since then.

So I'll take this space to mention a few.

First and foremost perhaps is the paper by Hanushek, et. al. This is another in a long line of papers by Hanushek that exploits a wonderfully rich dataset from Texas. Here is a quote from the conclusion.

Perhaps the most important finding is that peer average achievement has a highly significant affect on learning across the test score distribution. A 0.1 standard deviation increase in peer average achievement leads to a roughly 0.02 increase in achievement. Given that a one standard deviation change in peer average achievement is 0.35 of a standard deviation of the student test score distribution and that the use of lagged test score introduces error into the measure of peer achievement, the point estimate suggests that differences in peer characteristics have a substantial effect on the distribution of achievement when cumulated over the entire school career.

Second is a paper that examines data from Chile by Patrick McEwan. Here is the abstract:

This paper reports estimates of peer effects on student achievement, using a 1997 census of eighth-grade achievement in Chile. The data allow detailed measures of peer characteristics to be constructed for each classroom within a school. The paper addresses the endogeneity of peer variables by including school fixed effects that control for unobserved family and student characteristics. The estimates suggest that the classroom mean of mothers’ education is an important determinant of individual achievement, though subject to diminishing marginal returns. Additional specifications using family fixed effects are not suggestive that estimates are biased by within-school sorting. [The emphasis is mine]

So there papers find fairly significant peer effects, however a newer paper using arguably better data from Florida by Burke and Sass finds more modest peer effects:

We find that peer effects are not “one-size-fits-all,” but rather exhibit striking differences across students of different abilities and across different segments of the peer ability distribution. For example, the weakest students appear to experience the biggest positive impact from having higher quality peers. At the same time, however, such benefit appears to derive specifically from having peers in the highest quintile of the ability distribution. High ability students appear to experience the weakest spillovers from mean peer ability, but nonetheless may suffer sharp losses due to an increase in the share of peers of very low ability. The sizable effects observed in the nonlinear models are obscured in the linear-in-means models, within which we find only very modest, but positive, spillovers from mean peer ability. Furthermore, comparisons of effects between math and reading scores, and across different schooling levels, also depend on whether linear or nonlinear models are employed.

Considering the more nuanced results of the nonlinear models, the policy recommendations are not clear cut. For example, while low-ability students appear to benefit significantly from having top-quality peers, those peers will experience reductions in achievement gains from mixing with students of very low ability, reductions that may fully offset the weaker students’ gains. On the other hand, policies that mix middle and high ability students with each other are likely to strictly dominate those that segregate the top students in a separate track. While parents may prefer strict tracking, our results indicate that the highest-ability students actually benefit from mixing with students of middling ability. We also find that any negative peer effects from school choice programs are likely to be small. A choice program that attracted 2.5 percent students, all of them from the top ability quintile, would have only very small negative effects on the learning gains of lower ability student who remain behind.

So yes, peer effects are important, but the issue may be quite complicated in terms of how you sort and help low achievers without hurting high achievers. I think the lesson is that heterogeneity is a reasonable goal, but not a magic bullet to fix low achievement in schools and that a district has to weigh lots of factors when trying to find ways to improve achievement.  For example one way to ensure heterogeneity is to bus kids all over town, but then you potentially lower the neighborhood connection, parental involvement and most of all, spend a lot of money that may be better utilized in reducing class sizes or lengthening school years.

No comments: