One of the difficult problems in experimental science is drawing conclusions from a null experiment. Suppose a theory forbids an event to happen—faster than light travel for example is forbidden by the theory of relativity. Looking for faster than light travel and not succeeding is a result consistent with the theory. But, the result doesn’t “prove” the theory. Your experiment might not have looked for faster than light travel in the right way, in the right place, or at the right time. You could spend a lifetime looking for an event that shouldn’t happen, not see any examples, and still not know for sure if the event could never happen.
Contrast that situation with an affirmative experiment. For example relativity predicts the bending of light by gravity. Einstein became famous after astronomer observed the bending of starlight as it passed by the sun. That affirmative result doesn’t “prove” the theory either, but at least you know light bending by gravity is a real effect. You don’t have to keep looking.
I was thinking about the problem with the null experiment this week because it arises in the current debate about the effectiveness of the government bailout of the financial industry. This past September congressional leaders and President Bush predicted imminent collapse of the United States’ financial system unless a $700 billion bailout plan passed. On September 29, 2008 the House of Representative rejected the proposal and the markets responded with a 777-point loss in the Dow. It closed at the end of that day at 10365.
The economic catastrophe theory appeared to be confirmed. House members quickly saw the error of their ways and within days a new version—containing additional pork to assuage some hurt feelings—passed into law. So did the bailout work? Was financial disaster averted?
In the three and half months since its passage the Dow has fallen another 2000 points and is now near 8000. The unemployment rate rose from 6.2% in September to 7.2% at the end of December. Major banks such as Citigroup are still teetering on edge of collapse and asking for more government money. Had the federal government not passed a bailout bill in October these events would be cited as the predicted catastrophe. But, because the government did pass the bailout bill a different interpretation is needed. Advocates, including Henry Paulson, have argued that without the bailout the economy could be much worse.
Which brings us back to the problem of the null experiment. Would it ever be possible to conclude that the bailout hasn’t worked? As long as the Dow remains above zero and the unemployment rate less than 100% it will always be possible to say the economy could be worse. But just because a total economic collapse has not happened doesn’t mean the theory that the bailout worked has been proved. Saying that the bailout prevented a total economic collapse because a total collapse was not observed is faulty logic. Taxpayers could feed money to banks forever on the basis of that reasoning.
It’s time for treasury officials to devise some affirmative experiments. The needs are for actions designed to produce positive economic effects that can be observed and measured. Only then will we know if the money has been productively spent.