Economist's Notebook: Development Economists in the Laboratory

I have conducted research in a wide array of fields of economics: urban economics, industrial organization, labor economics, growth, international economics, environmental economics, political economy and public policy. But I am, first and foremost, a development economist. Like any academic discipline, economics has trends and fads that come and go and development is no exception. The latest trend is in randomized experiments. The leading institution for this work is MIT where the Poverty Action Lab supports studies based on randomized experiments. There is a growing debate about the value of such experiments, due, in my view, in large part to the zealous promotion of such research by its champions and their concurrent dismissal of natural experiments and other attempts to tease out causality from data.

The problem is, in essence, the fact that the world is complex and messy and as most human actions and interactions do not happen in a controlled environment, understanding the mechanisms that govern human actions and interactions is exceedingly difficult. Mostly we do as careful a job as we can, present as convincing a story as we can, but we never prove anything. However, in the standard empirical analysis, we are often dealing with data from a large population which makes generalizations relatively straightforward.

Randomized experiments, like for example, talking a set of schools in India and randomly selecting some to get computers and then studying the effects allows economists to (somewhat) control the environment and make a stronger causal statement. However, the generalizability of such trials is questionable. So too is the economic underpinning of the trial itself. Without having some ideal of the overall economic mechanism we might not really understand much at all. For example, do households respond to their kids access to computers in certain ways that has spillover effects that are positive or negative.

My take is that more data is always good, but that experimentalists (many of whom are the most influential voices in development economics) are wrong to dismiss more standard empirical work. I think experimental evidence is interesting, but often does not tell us much about the world in general. Empirical work based on theory is not always convincing but much of it provides good evidence to support or dismiss theoretical hypotheses and tells us a little more about the world in general. In other words, both have a role and a place in economics. What is distressing to me, then, is how much attention is paid to randomized experiments and how little attention is being paid to theory and related empirical testing. I don't think it is healthy to focus so narrowly on one approach.

I was happy, therefore, to read Steve Levitt's take:

A few months ago, Princeton economist Angus Deaton offered his vision for development economics.

In his piece, he rails against the movement toward relatively atheoretical, randomized experiments, calling for closer ties between theory and empirics. “The great economists should be trying to do something that is harder.” Now, in an excellent new paper, Harvard economist Guido Imbens fires back.

Imbens argues that Deaton is too dismissive of the special value that randomized experiments have in assessing causality, and that natural experiments, while not as good as randomized experiments, are far better than Deaton gives them credit for.
While it might seem difficult to mostly agree with both Deaton and Imbens, given that they espouse polar opposite views, strangely enough that is where I stand in this debate.

On the points Imbens makes, I think he is exactly right. However, what Imbens minimizes, in my opinion, is the importance of models and theory for motivating empirical research, including randomized experiments.

Given a question, of course you want a randomized experiment to give you the best answer possible. What I think has happened too much in economics recently is that the availability of experiments has trumped the asking of good questions. Or, put another way, anyone can do program evaluations based on true randomization, so why should some of the world’s best economists be devoting so much of their time to such exercises?

The great economists should be trying to do something that is harder.

Here is Poverty Action Lab's Abhijit Banerjee and Esther Duflo's take.