Economist's Notebook: Small-Sample Bias and the Media

On OPB last week Ethan Lindsey reported on the firing of a reporter for the Bend Bulletin wherein it was suggested that housing industry poo-bahs may influence the Bulletin’s coverage of the housing market. There was also complaint, echoed elsewhere, that recent press coverage of the subprime housing mess has been too one-sided and has caused increased deterioration in the housing market. This got me thinking - what is the media's responsibility when it comes to sending a signal about markets?

Is it the responsibility of the media, for example, to get the right message across about probabilities when covering accidents? Suppose a fiery car wreck with a large color picture is on the front page of a newspaper, it is the responsibility of the paper to include many pictures of cars making safe journeys and include in the story anecdotes about many other persons who did not crash? I don’t think so. But this appears to be what home builders, realtors and mortgage lenders are complaining about now in the media’s coverage of the housing market. People think the situation is much worse than it is, they claim, due to negative press attention.

But people make mistakes of inference like this all the time: it is what economists call a small-sample bias. Technically, this is where a set of observations are too few to get an accurate average measure from the sample. Suppose I fly to Nashville and find that it is quite cold on the day I arrive in mid-March. Based on that one observation I might conclude that March in Nashville is very cold. But then the next three days are warm and I realize my inference might have been wrong – I concluded too soon. If I spent many years in Nashville, I would probably be able to make a guess about the average March temperature that was quite accurate.

This is similar to what happens when we read a story of a person loosing money on a house – we tend to give the one observation too much credence. But this works both ways, during the housing bubble, stories of people making a lot of money buying and selling houses caused people to conclude that it was almost impossible not to make money.

Do newspapers have some sort of responsibility to respond to these errors of inference by always pairing one bad with one good story? I think not - it is up to readers to understand the errors they are making.

It is worth noting that it might not be an error to see a single story about someone who made a lot of money buying and selling a house and believe that this informs them about the state of the world. If lots of other people read the story and making and gain confidence which leads to investments in property, then the inference based on this single story might turn out to be correct. It is the story itself that made it so. In this sense news coverage is endogenous to the market it can move the market and market movements can cause coverage to appear. It is all part of the bubble process.

What is the answer? Good economics and econometrics training for everyone of course! Look at data, not anecdotes, when making inferences and try and look at data in ways we know eliminates or minimizes biases. Absent that, try and understand that anecdotes are not good ways to make inferences about the state of the world.