Tag: program evaluation

Bill Gates on Accountability and Program Evaluation

In today’s WSJ, Bill Gates writes about the power of data collection and analysis.   He writes; “In the past year, I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal …”

I have been told that before Gates dropped out of Harvard that he audited first year Ph.D. econ classes there.  If he takes his goal seriously, he will need to grapple with some frontier research questions.   All micro econ students are taught the production function. In the typical boring example, a firm hires labor and capital to produce pizza.  The professor stands at the board and writes; pizza = f(L,K) and talks about the demand for inputs where f(L,K) is the production function.  What makes this boring is that the firm knows f(L,K) and inputs are homogeneous.  All workers (L) and all robots (K) are identical.

But, diversity and uncertainty are the spice of life and of modern economics!   For a taste of this, read Sherwin Rosen’s AEA Presidential Address.  What Gates’ article is really about is how from observing some noisy measures of output, and inputs do we infer what the production function f() is?  Gates doesn’t care about “pizza production”.   He cares about healthy kid formation and human capital formation but on some level it is the same question.   Consider the following example:

Continue reading “Bill Gates on Accountability and Program Evaluation”

Concerning replication

Heraclitus said that no one can step twice into the same river. By the same token, it isn’t really possible to do the same program in two different places.

I’m in London for meetings on criminal justice organized by the Centre for Justice Innovation (the UK arm of the Center for Court Innovation) and Policy Exchange. A small meeting that ran for two hours today tried to figure out whether the coerced-sobriety approach of the South Dakota 24/7 project (the alcohol version of HOPE) could be used in Scotland, with social problems, customs, and institutional arrangements quite unlike South Dakota’s. (In particular, Scotland has no such thing as a two-day term of confinement.)

The answer seems to be that you could try to do something based on similar principles and with similar aims, but you couldn’t really do the program in its trademark style, and even if you’re convinced 24/7 works on drunk drivers in Sioux Falls you can’t be sure that the alternative version would work on drunken wife-beaters in Glasgow.

That suggests a more general reflection. Heraclitus said that no one can step twice into the same river. By the same token, it isn’t really possible to do the same program in two different places.