It’s Time to Call It a Failure

Steven Horwitz

Recently Mario Rizzo gave us permission to say “we told you so” with respect to the failure of the fiscal and monetary stimulus policies that have been the predominant policy response to the Great Recession.  I think he was quite right to do so, and also quite right to note that many of us predicted that stimulus advocates would argue, in return, that it simply wasn’t big enough to do the trick.

That claim is, of course, not falsifiable.  Those advocates would claim, perhaps rightly, that the arguments of stimulus critics that things would have been better without the stimulus is equally non-falsifiable.  Fair enough, although we may have some markets in which we can get a sense for what might have happened.  But there is one thing we can do, which is to look at the actual results of the actual policies that were adopted and compare them, at least in one case, to the promises and alternative scenarios that the promoters themselves put forward at the time.  While this cannot conclusively rebut the argument that “it wasn’t enough,” we would have good reason to be skeptical of that argument if what we got did not improve things, nor come near to doing what its advocates promised.

First let’s look at the unemployment and inflation rates in January of 2009 when the concrete proposals for a major fiscal stimulus were in the air.  That January the unemployment rate was 7.8%.  In May 2009 it topped 9%.  In the 24 months since then, it has been below 9% only twice, and then by one or two tenths of a percentage point (and that may have been due to discouraged workers leaving the labor force).  In the aftermath of the stimulus, even given a few months lag for the spending passed in February to kick in, we’ve seen two solid years of unemployment of essentially 9% or higher.  In the early 80s, we saw 21 months of 8.8% unemployment or higher. May 2011 makes 26 months of 8.8% or higher. Finally, the May 2011 unemployment rate was 9.1% or 1.3 percentage points higher than when the call for stimulus was being made.

Of course the Obama Administration argued that the stimulus plan would lead to unemployment peaking at about 8% in the third quarter of 2009 and that it would be around 6.75% this summer.  Not only that, they argued that if we did nothing, that is, if we passed no stimulus plan whatsoever, the unemployment rate would peak at about 9.0% for most of 2010 and would be at about 8% this summer. 

In other words, with the stimulus, we saw an unemployment rate that peaked a full percentage point higher than what its promoters said we’d have without it, and that has remained that much above the no-stimulus prediction since then.  By its own standards, the fiscal stimulus has been a failure. 

On the inflation side, matters are a bit more complicated.  The inflation rate in January of 2009 was effectively zero and the rate for the whole year that followed was negative as the depths of the recession set in.  But since then, the CPI has turned positive again, with the May 2011 annual rate being 3.57%, up from 1.63% just four months earlier in January.  That 3.57% annual rate is higher than the average annual inflation rate (measured by the CPI) in every year from 2000 to 2007.  Arguably that period was seeing inflation, but mostly in assets that were not picked up by the CPI.  This time around, the CPI may well be picking it up, thanks to the fact that the transmission path is now affecting consumer goods more directly.  Those who warned that the quantitative easing would eventually produce inflation on the consumer side have some justification for now saying “I told you so.” 

If you want to play the misery index game (unemployment plus inflation), it was 7.8 in January of 2009 and it is now 12.7.  Other than being a tad higher in December 2009 (again, after the stimulus), it has not been that high since the end of the back-to-back recessions of the early 1980s in May of 1983.  If we work from May 2009, when it was 8.1, the effect of the stimulus in the succeeding two years has been to drive the misery index up by over 50%.

The effects on real Americans?  As one measure, let’s look at the labor force participation rate.  In January of 2009 it stood at 65.7%.  In May of 2011, it was 64.2%.  Again, you’d have to go back to the end of the recessions of the early 1980s to see a participation rate that low.  It’s also worth noting that it was falling before January of 2009, having peaked in early 2007.  The stimulus did nothing to halt the decline and possibly accelerated it as the persistent 9% unemployment rates have led to more workers choosing to drop out of the labor market altogether. 

I could go on, but it’s really not necessary.  The current administration came into office promising that these stimulus policies would save us from economic disaster, and by their own definition of “disaster” (namely, a year’s worth of 9% unemployment), they have failed.  The stimulus has produced an economy worse than the disaster they predicted without it.

Given that this administration also promised that it would, unlike what it claimed of the Bush years, rely on data and science to make policy, it should be admitting its own failure.  The data in this case tell a pretty clear story of failure, and failure in comparison to the administration’s own criteria. 

There are those who will continue to say that the real failure was that the stimulus was not big enough.  These data will surely not persuade them, but they should.  It is not clear by what economic logic it could be true that some stimulus will make matters worse but lots will make matters better.  It would be one thing if the actual stimulus had improved things a little, then the case for claiming that a larger stimulus would do more would be at least plausible.  But when a little makes things worse than a “disaster,” it boggles the mind to think what kind of real disaster more would bring.

It’s time for those of us who said the fiscal stimulus and quantitative easing were bad ideas to stand up and proudly say “we were right and the data are on our side.”  It’s time to call it a failure.  Period, end of sentence.

Comments

[Daniel Kuehn]

Why should anyone put any stock whatsoever in a counterfactual produced by a politically appointed team extremely early on in the recession?

A fail to see why people are so fixated on this.

Why are you outsourcing good scientific legwork to politicians, Steve? Would you be writing this if they had produced a "without the stimulus scenario" that was closer to the expectations of stimulus advocates outside the administration who described the administration's numbers as "a lot more optimistic" than their own?

What is it about a document marketing the stimulus, produced by a Democratic administration, without an explanation of the forecast, produced in the early stages of the downturn in the real economy that you find so convincing that you're willing to make a claim like this???

I simply don't see it.

[Daniel Kuehn]

It's not even that their politics particularly bother me, obviously, and it's not like I think Romer and Bernstein were up to anything particularly nefarious. I just have no idea why you think - of all the reams of data, estimates, and work that's been done on this recession - that this should somehow be a benchmark of anything.

[Mario Rizzo]

The points are (1) The burden of proof must be on anyone who advocates a huge increase in government (spending); (2) The original forecast was, in fact, taken seriously by economists; (3) Some kind of forecast is necessary to answer the question, if only probabilistically, whether fiscal stimulus "worked."

And Mr. Kuehn is getting quite tiresome.

[Daniel Kuehn]

That's unfair Mario. I know and agree we need some kind of counterfactual to evaluate the question. This is precisely why I posed my question to Steve - what is it about THIS counterfactual that he finds so impressive?

I honestly don't know.

You write regularly criticizing economists who fancy themselves as scientists and who think they can make precise claims about the economy. If you're going to call me "tiresome" you at least owe me an explanation of why you apparently feel differently about the Romer-Bernstein forecasts and why you think these are reasonable counterfactuals.

[Richard Ebeling]

I fear that the "debate" over the "stimulus" spending will now go on for years and decades. Just as it has gone on for decades whether the Great Depression would not have been worse, if not for the degree and forms of New Deal "stimulus."

Take the related issue of whether the lack of private sector recovery and job creation since the bottom of the current recession has been significantly impacted by what Bob Higgs has hammered away at as "regime uncertainty." That issue will not be "settled" in any foreseeable future, either.

By the way, the impact of regime uncertainty was argued back in the 1930s, as well. For example, Thomas Nixon Carver, a long-time professor of economics at Harvard University, a strong classical liberal and one of the formulators of marginal productivity theory at the end of the 19th century, wrote in 1935:

"In every nook and corner of the the land there are men and women of vision and enterprise, eagerly watching for chances to start some productive business [who were being inhibited by a lack of confidence]. . ."

"Confidence cannot be restored until fear is removed. We cannot remove fear unless we know what men are afraid of.

"What are men afraid of? Some are afraid that government will:

"1. Experiment further with our money.

"2. Impose high taxes on industry

"3. Suppress or interfere with enterprise.

"4. Start state-supported industries to compete with the industries that are taxed.

"5. Be captured by radical politicians."

And, thus, the private sector was reluctant to restore production, rehire, and undertake new investment of these "uncertainties" of government policy.

(Thomas Nixon Carver, "What Must We Do to Save Our Economic System?" [May 1938], p. 78.)

Alas, the answer to these questions rely upon the theoretical "glasses" through which one sees the causal and inter-related factors at work, and the presuppositions one holds concerning the nature and workings of the market vs. the government, in general.

And unanimity on these things are not likely in the near future.

Richard Ebeling

[Daniel Kuehn]

I completely agree Richard. These are complex systems we're dealing with. If the answer were that easy we'd all be happy technocrats by now.

This isn't to say good work can't be done. We profitably study complex systems all the time, and some arguments are more persuasive than others. But a hurried forecast produced in the midst of a constantly changing environment by an embattled team that hadn't even come into office yet is not a legitimate foundation for the inquiry.

[Sandre]

"If I can't fix the economy in three years, you can call me former president" - President Barack Obama in 2009.

If Obama is a man of his word, he has got a choice to make - drop out of the 2012 presidential race.
http://www.youtube.com/watch?feature=player_embedded&v=SmRgaKfWMPA

[Steve Horwitz]

It's not a matter of being impressed Daniel, it's a matter of it being the ones that *those who made the case for the stimulus put forward as justifying the stimulus spending*. I'm judging them by the criteria that THEY IMPOSED UPON THEMSELVES. I make no judgment how right or wrong it was per se, only that it's what THEY said would happen and the outcome was worse than what they said would happen if we did nothing.

By their own criteria, they have failed.

[Mario Rizzo]

Steve's point explains why even "moderate" economists like Greg Mankiw have periodically reproduced this about graph. This was the test that the Administration's economists implicitly set out. (I say implicitly as a "test" -- it was then a prediction.) It was part of the basis upon which the stimulus was sold to Congress.

Now, of course, if we are to deal with the issue of whether the stimulus was beneficial the matter becomes much more complex. We have to choose a standard of "benefit." And then we need an economic theory which is adequate to the task of making the empirical claim.

The problem is that we cannot use a theory that presupposes the very Keynesian relationships we are trying to test. Many theories do not permit clean tests. For example, the rational expectations macro-models are joint tests of the efficiency of expectations AND the particular theory of the economy's structure.

I am not a macroeconomist. I frankly do not know how to go about making the "definitive" test. But, in my reading, it does not seem as if the macro profession is settled on how to go about this either.

I think economists and economics are better suited to dealing with what makes for good long-run policy than with the issues of counter-cyclical policy.

That is why I believe that we must stick to what we know works in the long-run to promote stability of expectations, economic growth, etc.

One of the problems with macroeconomic policy is that the policymakers go from one short-run attempt to make things better to another (eg, Fed fine tuning, so-called). We also go from one bubble to another. Policymakers also try to improve on the market allocation of capital through government-sponsored institutions like Fannie and Freddie.

It is not easy to keep one's eyes on the long-run because we all live in the short. But reality is what it is.

[Bob Murphy]

Daniel, in this article I pointed out that Mark Zandi in late 2008 made a forecast that said without stimulus, unemployment would top out at under 9% in October 2009, and GDP would fall 2.2%. Oops, it turns out that with the stimulus (which was less than what Zandi called for), unemployment broke 10% and (real) GDP fell 2.6%.

And you can't even say that Krugman thinks Zandi is a moron--Krugman was praising Zandi's modeling done in June 2009, because it correctly predicted the rate of change of GDP growth in relation to stimulus spending.

Please point us to the Keynesian forecasts made in late 2008, explicitly saying that without stimulus, the economy would have (say) 12% unemployment, and real GDP would fall (say) 4%. It's not enough to cite Krugman saying "Romer is being very optimistic," because I've argued elsewhere that his own back-of-the-envelope calculations are nowhere near to predicting what happened.

[Daniel Kuehn]

Bob Murphy -
Forecasts of complex systems are inherently unreliable. To think otherwise is a fatal conceit. Forecasting oughta be done because it's better to try something than fly blind. But I think of macroeconomic forecasters like a think of weather forecasters. God bless 'em. They do good work. But I'm not going to test meteorological models by taking a forecast that the weather channel made several weeks ago and test it against what actually happened.

Who WOULD do that, Bob? Who has EVER treated macroeconomic forecasting like that?

I don't see the point in answering your question until I know that, because until I know that your question doesn't make all that much sense.

[Daniel Kuehn]

And speaking of this point that I made to Bob, and the point that Steve made earlier about this being "the standard they set for themselves"...

... is it the standard they set for themselves???

Do you have any reason at all - any whatsoever - to think that in January 2009 either Romer or Bernstein would have said "a year and a half from now the way we should test the stimulus is by comparing it to this best-estimate forecast we pushed out the door at the very beginning of the crisis"?

Christie Romer - whose work on fiscal policy and the business cycle is second to none in plausibly identifying the model and finding a counterfactual - do you REALLY think this is "the standard she set for herself"?

Do we have any evidence that that is the case?

And to reiterate my question to Bob - has anyone ever understood forecasts in this way?

[Daniel Kuehn]

*wow - I mean two and a half years, of course.

[Bob Murphy]

Daniel, I'm sorry to say this since you and I are internet buddies, but you seem to me to be a clearly moving target on this. Your first reaction to Steve's post above said this:

"Why should anyone put any stock whatsoever in a counterfactual produced by a politically appointed team extremely early on in the recession?

A fail to see why people are so fixated on this.

Why are you outsourcing good scientific legwork to politicians, Steve? Would you be writing this if they had produced a "without the stimulus scenario" that was closer to the expectations of stimulus advocates outside the administration who described the administration's numbers as 'a lot more optimistic' than their own?"

In response to Mario's response to you, you then said, "That's unfair Mario. I know and agree we need some kind of counterfactual to evaluate the question. This is precisely why I posed my question to Steve - what is it about THIS counterfactual that he finds so impressive?"

So you clearly were saying that you (a) agreed a counterfactual is necessary, but (b) you can't understand why we Austrians keep picking this particular counterfactual. Why don't we look to Keynesians who weren't working in the political realm?

So, I pointed to one prominent Keynesian forecaster who botched things just as bad as Romer--and whom Krugman later praised as being right on the money with his modeling (when the results weren't an embarrassment to Keynesianism). I then asked you to point me to a Keynesian forecaster who did NOT botch things up.

And your response now is, "Forecasting is difficult."

So you are a moving target, and it seems pointless to continue debating this point.

[Daniel Kuehn]

What do you mean "now" my response is, Bob.

When have you ever known me to argue that the role of forecasting is to provide a counterfactual to analysts two and a half years later.

I concurrently hold the positions that:

1. Politically entangled economists, while in most cases well-meaning, are not always the best to look to.

2. We need plausible counterfactuals to make assessments.

3. Forecasts are not counterfactuals. They're an effort at groping in the dark because that's all we have.

What is contradictory in these three points? How am I a moving target if I've always thought these things? When have you ever known me not to think these three things?

re: "I then asked you to point me to a Keynesian forecaster who did NOT botch things up."

And I thought I made clear that no forecast is up to the task that Steve sets it to, and no forecaster is under the impression that it IS up to such a task. So I suppose my answer is I can't point to one (although I think "botch things up" is a harsh way of describing the inevitable vagaries of macroeconomic forecasting).

Now I think I asked you a question too. Who has ever suggested that forecasts should be used this way?

[Bob Murphy]

Daniel wrote:

Do you have any reason at all - any whatsoever - to think that in January 2009 either Romer or Bernstein would have said "a year and a half from now the way we should test the stimulus is by comparing it to this best-estimate forecast we pushed out the door at the very beginning of the crisis"?

Really, Daniel? No reason "whatsoever" for us to think that? OK, but then Romer is a big fat liar to the American people. What the heck were they doing, putting that graph into a political document that the government would use to get people to support the stimulus?

You are making it sound like Romer was into her fifth gin-and-tonic on a Friday night at the bar, and Wolf Blitzer had his cell phone set to record unbeknownst to her as he asked what she thought unemployment would be without the stimulus. No, that's not the context of her graph.

[Daniel Kuehn]

re: "Really, Daniel? No reason "whatsoever" for us to think that? OK, but then Romer is a big fat liar to the American people."

Let me rephrase it this way, Bob.

If Romer really thought what Steve is trying to do here is legitimate in any sense, why didn't she use forecasts as her counterfactual in the tax ependiture paper she wrote with her husband? Why did she put all that painstaking effort looking into the Congressional record to properly identify her model when she could have just used earlier forecasts by forecasting firms.

If she really thought that what Steve is proposing here made sense, don't you think it would have been easier for her to use forecasts rather than establish exogeneity the way she and David decided to establish exogeneity?

[Daniel Kuehn]

And we don't have to restrict this to Romer, of course.

Are you aware of a single analysis that uses past forecasts as counterfactuals.

I am not, but every commenter here so far has been in the business longer than me so perhaps you know of one.

Is there such a paper? If not, why not? Could it be that this is not a defensible way to evaluate the question?

[Martin]

Didn't you also tell us so that interest rates would rise and inflation would rise due to monetary stimulus? I have the impression you are being rather selective.

Not that I do not find the rhetoric to be impressive, but you are just preaching to the choir. I don't see how this is supposed to be convincing to anyone outside the libertarian penumbra.

[Mario Rizzo]

A relevant article in today's New York Times:

"The pace of economic expansion has repeatedly fallen short of the Fed’s predictions."

http://www.nytimes.com/2011/06/22/business/economy/22fed.html?_r=1&hp

Is a prediction a standard of success? Does this mean that the Fed "failed" by its own standard?

However one answers this question it does show the limits of counter-cyclical policy.

[Current]

A few points...

I agree with Martin's criticism of Bob Murphy. I get the impression that Bob started the crisis believing that the velocity of money or demand for money is fixed, and therefore any creation of new money would cause price inflation. I'm not sure he has changed his view, but it's not one I agree with. I don't think Steve Horwitz agrees with it either or made the same sort of predictions of high price inflation at that time. Many, maybe even most Austrian Economists don't accept the Rothbardian view on money and the business cycle.

Daniel is confusing the question of the accuracy of economic predictions with a different question. When formulating economic models and theories it obviously makes sense to construct counter-factual scenarios using data from past history. Then from that we have sets of theories that rival schools of economics propose. Now, to test those theories I see nothing wrong with comparing their predictive abilities, which is what Steve is doing here. The point of having a fairly finished theory is that it allows users of it to make predictions without having to go back to the process of theory-construction. That's why it's appropriate to criticise those who use theories to build predictions on the accuracy of those predictions.

Lastly, Mr.Kuehn seems to get some credibility around these parts for being an honest debater. Over on his blog I posted a comment on how various different interest rate theories stack up against each other. It was much more on the side of nerdy than vitriolic, I criticised nobody in it. But for some reason it disappeared, funny that.

[Daniel Kuehn]

re: "Now, to test those theories I see nothing wrong with comparing their predictive abilities, which is what Steve is doing here."

No, Steve is not doing this. He has offered no comparable forecast from Austrians. Furthermore, he hasn't noted Keynesian claims about when growth would fall off - early 2011. This seems relevant to point out, since predictions of change are somewhat more viable than predictions of levels.

re: "It was much more on the side of nerdy than vitriolic, I criticised nobody in it. But for some reason it disappeared, funny that."

I can check the spam folder - I believe I rescued one of your comments. Blogger has a habit of identifying legitimate comments as spam. I'll check again now. I've only deleted a handful of comments which were verbally abusive to myself or others. I don't recall you ever posting anything like that or deletin anything you've written.

[Daniel Kuehn]

And I should note - we test scientific theories by confronting the data with the theory. This does NOT necessarily imply that we test forecasts.

We don't rely on predicting future evolutionary paths to verify evolutionary biology.

We don't rely on the accuracy of meteorological forecasts to test climatology.

We don't rely on predictions about where the next earthquake will occur to verify seismology.

When we deal with complexity, we confront the theory with the data by explaining what we have observed - not be forecasting and checking our forecasts. No economist does that. None. We forecast because it is pragmatic to make decisions on the basis of an educated guess. It does not play a substantial role in economic science and theory verification.

[Daniel Kuehn]

I recognize I sound obtuse, but this is not trivial stuff.

I'll blink when someone can point to an instance where an impact estimate in the economic literature was determined by comparing actual data to an ex ante forecast. I'll concede when someone can show me multiple, well-received cases of this identification strategy.

Until then, I think I've been talkative enough on this point.

[Current]

"I can check the spam folder - I believe I rescued one of your comments"

It's reappeared now, thanks for that.

[Steve Horwitz]

Once again, I am judging the stimulus as a failure by the very criteria that those who were supporting it (whether politicians or the economists they employ) used to argue it was necessary. That's all.

(One might use a similar strategy for judging the failure of US foreign policy I might note.)

Me thinks Mr. Kuehn doth protest too much.

[Daniel Kuehn]

Steve -
1. It's still not clear to me why you're under the impression that this forecast is the standard by which they expected the stimulus to be judged rather than... well... a forecast. I've given lots of reasons why I doubt this. Why do you think they think it's the standard?

2. Clearly many of the commenters are making broader claims than you did initially, and you'll see in most cases I'm responding to other commenters.

3. You wrote "it's time to call it a failure". You can understand why some people might mistake you for saying it is a failure by your standards, rather than just the administration's standards.

[Troy Camplin]

Everyone complains about the weather, but nobody does anything about it.

Now, why is that? Could it be that the weather is a complex system that in in fact uncontrollable? When the weather man predicts the weather, and doesn't get it exactly right, we are not surprised because 1) he is discussing a complex system, and 2) we know he had nothing to do with the outcome anyway. It would no doubt be different if he said, "We see a front coming in, but it's going to rain in Z rather than X, but since X has had a drought for a year, I am going to go out and seed the clouds so that it will rain in X," and, after he did so, it rained in Y instead, which caused flooding, because it had been raining there a lot lately. Would we accept his, "Well, weather is a complex system . . ." argument? No. We would say, "Of course it's a complex system -- that's why you shouldn't be messing with it, you moron!"

One can make the same argument about evolution. Of course we can't predict the outcome of evolution. But nobody is fool enough to predict it, and nobody is idiotic enough to try to direct it, either.

Sadly, none of this is true of the economy, which is more complex than either weather or biology.

But Daniel is trying to have it both ways. Either the economy is so complex that one cannot predict what will happen and interventions are inherently unpredictable in their outcomes, or we can knowledgeably make Keynesian-type interventions of whose outcomes we are fairly confident.

Let me throw out a biological example. Let us say that you have cancer. What is the cure? Chemotherapy -- which is a fancy way of saying, a cellular poison we hope will kill the cancer before it kills the body. With chemotherapy, you are purposefully poisoning the body to kill the cancer. You give as much as you can so that you don't kill the patient. Now, if such chemotherapy doesn't work, do we give even more? Of course not, because then you will kill the patient. A little poison may kill the cancer, but a lot of poison will kill the patient.

Or, if you prefer, consider the use of aspirin. If I have a headache, I take one so that the headache goes away. But if I take a lot, does that mean the headache will go away faster? No. You die.

Now I am sure that Daniel will find all kinds of faults with my examples. They are not a perfect 1:1 correlation with economic interventions, and are not meant to be. Rather, they point out that, if anything, drugs to cure a disease must be given in small doses to work, not big ones. If the drug in question does not cure the disease, it is likely that it was the wrong one for the disease, not that we should be giving it in ever-greater doses.

[Lee Kelly]

Daniel,

I think the underlying objection of Horwitz and others is that the forecasts, which are really just predictions derived from the same theory under different initial conditions, were wildly wrong.

Maybe that's because the initial conditions changed in some unexpected way -- the economy is complex, after all. It does not indubitably follow that the theories used to construct these forecasts are false.

However, this is not an isolated occurrence. These type of forecasts seem to be wrong most of the time. The real "I told you so" has to do with the pretense of knowledge, the fatal conceit, or whatever you want to call it.

I don't like much of Steve's post, but he is not wrong about this one. Economists need to learn some humility from these types of experiences; those that don't are little better than charlatans.

[Current]

"We don't rely on the accuracy of meteorological forecasts to test climatology.

We don't rely on predictions about where the next earthquake will occur to verify seismology."

Seismologists don't present themselves as being able to make clear predicts. They also don't claim to be able to clearly predict the impact of certain changes. The Keynesians Steve and Mario have quoted have done exactly that.

There is a difference here between "science" side economics and "policy" or "engineering" economics. Keynesians, or at least some Keynesians, claim to be able to do the later. They may not claim to be "like dentists" but they claim to be something closer to that than we do here. That's why I think it's reasonable to measure their success by the success of their predictions.

Suppose a Seismologist said "I think there will be a richter 6 earthquake in LA between 2012 and 2013, unless some water is pumped into this fault line in which case it will be richter 4". In that case wouldn't other Seismologists be critical if no earthquake occurred?

If economists don't think their knowledge is robust enough to support predictions then they shouldn't make them.

[Lee Kelly]

"But Daniel is trying to have it both ways. Either the economy is so complex that one cannot predict what will happen and interventions are inherently unpredictable in their outcomes, or we can knowledgeably make Keynesian-type interventions of whose outcomes we are fairly confident." - Troy Camplin

But Daniel can have it both ways. There is clearly a middle ground on this issue; predictions can be accurate without being precise. The problem is that many economists present a veneer of precision; they are not content to just say that something will increase or decrease. Instead, they create the illusion of being able to predict rates of unemployment like astronomers predict eclipses. And eager to see their preferred policies put into action, they fail to emphasise the many qualifications that should always accompany such forecasts.

[Robert Roddis]

I think that it is essential to always insist that statists demonstrate conclusively that:

a) There is a serious problem that cannot be solved by free market relationships; and

b) There is a statist solution that not only can solve the alleged problem but that the statists can prove that this is so beyond a reasonable doubt.

The burden of proof must always be on the statists to justify their interference in people’s lives.

In the instant debate, the statists have not even attempted either demonstration.

[Daniel Kuehn]

Lee -
re: "I think the underlying objection of Horwitz and others is that the forecasts, which are really just predictions derived from the same theory under different initial conditions, were wildly wrong."

Some people have also alluded to this.

It's been years since I've taken time series or forecasting, but if I recall I don't think the major approaches to forecasting are based on Keynesian models. It's fancy, less naive versions of "what happened yesterday plus a little new information is likely to happen tomorrow."

Forecasters are welcome to provide details on this, but that's my understanding of how it is done.

What theory are you under the impression these forecasts are derived from?

re: "Economists need to learn some humility from these types of experiences"

PRECISELY. Forecasts should not be held up the way Steve is holding them up. Romer and Bernstein never made the assertion (to my knowledge) that their forecasts could bear the load that Steve is placing on them.

[Daniel Kuehn]

What is based on a Keynesian model is the gap between the two projections in the Romer-Bernstein model.

They take prior theory and empirics and figure out a multiplier and apply that multiplier to the baseline forecast.

But I'm not aware of any approach to forecasting that relies on Keynesian models. I would be interested in hearing why anyone thinks the contrary.

[fundamentalist]

Steve: “I am judging the stimulus as a failure by the very criteria that those who were supporting it (whether politicians or the economists they employ) used to argue it was necessary.”

Exactly! I am confused about what Daniel’s objection with Steve’s post is. Is Daniel dredging up some perceived inconsistency between this post and past ones, or is he saying that we shouldn’t point out the failures of mainstream economists?

And why should we not compare the weather forecasts by their accuracy? Everyone does it in their minds and everyone knows they’re incredibly bad. That’s why we pay little attention to them.

Economic forecasts are important indicators of the soundness of theory. Mainstream economists have made modeling of empirical data the sole method of confirming theory. So should we not test the forecasts of the models to determine their accuracy?

Forecasting failures won’t change the minds of the Krugman’s of the world, but for those still trying to make up their minds as to correct theory will be influenced.

And top economists made the models and forecasts. Politicians were only the pimps.

[Lee Kelly]

"What theory are you under the impression these forecasts are derived from?" - Daniel

I had assumed they were derived from some kind of Keynesian-esque model. Perhaps not. In any case, there remains an implicit hypothesis, because nothing about the future is implied by past data.

"Romer and Bernstein never made the assertion (to my knowledge) that their forecasts could bear the load that Steve is placing on them." - Daniel

Then why aren't they emphasising this? Why are the forecasts so precise? If the forecasts can't bear the load that Steve is placing on them, then how could they possibly bear the load of guiding public policy? If what you are saying is true, then someone is being very misleading, whether intentional or not.

[von Pepe]

What Blinder and Zandi are reporting is the Keynesian theory that was built into the model.

http://econlog.econlib.org/archives/2010/07/how_the_blinder.html

I always love how dk says, "I am not aware of any..."

Well, if I had a nickel for all the things dk is not aware of...

[Mario Rizzo]

We all need to distinguish in the realm of theory:

1. The explicit test of a theory as put forth by its proponents.
2. The implicit test based on the logic of the proponent's argument.
3. The "correct" test of the theory.

Now if the realm of policy we have to consider:

1. Does the policy follow from the theory in fact, not just in the opinion of the proponents?
2. What is the policy supposed to do -- that is, what goal(s) did the policymakers announce?
3. How do we determine if the goals have been met?

On the last we cannot *presuppose* the validity of the theory from which the policy is supposedly "derived."

What is needed is a theory that, by and large, has done well in explaining previous cyclical events -- where "explain" means whatever the analyst's philosophy of science impells him to.

Now, if the Great Recession is "unprecedented" as many economists were claiming a couple of years ago, then it might be very hard indeed to answer the policy effectiveness question.

[Daniel Kuehn]

fundamentalist -

re: "And why should we not compare the weather forecasts by their accuracy? Everyone does it in their minds and everyone knows they’re incredibly bad. That’s why we pay little attention to them."

This is my point!!!!!

Now let's take this one step further - do we use this to conclude that we don't have theories that adequately explain weather? Of course not. Because there's a difference between empirical verification and forecasting.

re: "Mainstream economists have made modeling of empirical data the sole method of confirming theory. So should we not test the forecasts of the models to determine their accuracy?"

To improve forecasting - yes. To test theory - no.

[Daniel Kuehn]

Lee -

re: "I had assumed they were derived from some kind of Keynesian-esque model. Perhaps not."

So my understanding is that it boils down to regressing on lags. Now, there are corrections. You can build that into a structural model. But there's nothing especially "Keynesian" about any of that. A good example is the Heritage Foundation analysis of the Ryan plan. They use the exact same forecasting model that Keynesians use. There was nothing particularly Keynesian about the forecasting model itself.

[Daniel Kuehn]

von pepe -
Right - read the link. Step 1 isn't particularly "Keynesian". Step 2 says you make the model match actual data. Step 3, 4, 5, and 6 basically say "rerun it all without your assumptions about what the stimulus did - which are presumably based on PAST empirical work (like Romer & Romer) and PAST theory".

So the baseline - step 1 and step 2 - has nothing to do with any Keynesian theory. This is why Heritage is using the same models that Keynesians are.

As I said above, what uses Keynesian theory to predict results is the gap between the baseline without stimulus and the baseline with stimulus. The baseline itself is just a fancy version of "regress y(t) on y(t-1)".

[Steve Horwitz]

Whatever anyone else has said, I have said NOTHING about "Keynesian" models or Austrian predictions or anything else.

All I am saying is that the people who were telling us we needed this stimulus or we'd have the "disaster" of a year of 9.0% unemployment were wrong. Their argument that the stimulus would improve things over what they claimed would happen without it was not only wrong, it was backward: matters are worse with the stimulus than without it, according to their own criteria.

The case made by this administration and the economists and others who provided the arguments to support it was wrong. Their proposed solution not only failed to make things better, things are worse than they said it they'd be without it.

Why is this simple point so hard to understand? Someone please refute the argument by showing how the stimulus produced the results its proponents promised, or at least didn't make things worse. If you can do that, then we'll have a conversation here.

[Lee Kelly]

Mario,

If the theory in question, in conjunction with some initial conditions and auxiliary assumptions, merely describes how the world functions, then how can a policy follow it? Policy proposals have implicit normative assessments of the situation -- how it is and how it should be different. A policy response never follows from a theory unless its proponents have smuggled in value judgements.

Policy responses can be understood as attempts to alter initial conditions to bring about another outcome. The problem, as I see it, is that even if the theory in question happens to be true, one cannot just change the initial conditions with the clap of one's hands. The "initial conditions" are the plans and actions of millions of people, each responding to interventions in unpredictable ways.

The degree to which this problem inhibits or utterly prevents successful intervention differs from one circumstance to another. The most effective "interventions" do not come from wise bureaucrats, but from the evolution of traditions, conventions, and institutions, because only these types of decentralised process can possibly cope with the complexity.

Okay, this comment got a little off track.

[Mario Rizzo]

"A policy response never follows from a theory unless its proponents have smuggled in value judgements."

Yes, I agree -- of course. But I was taking it as a given that the goals were to reduce unemployment, increase aggregate output and the like.

John Neville Keynes distinguished correctly among the science, art and ethics in economics. All three are involved in policy.

[Lee Kelly]

"Whatever anyone else has said, I have said NOTHING about "Keynesian" models or Austrian predictions or anything else." - Steve

It's implicit in everything you wrote. For example:

"people who were telling us we needed this stimulus or we'd have the 'disaster of a year of 9.0% unemployment were wrong. Their argument that the stimulus would improve things over what they claimed would happen without it was not only wrong, it was backward: matters are worse with the stimulus than without it, according to their own criteria."

The "people" are mostly Keynesians of some colour. Their "arguments" appealed to Keynesian models and principles. Even if the specific forecasts were not derived with the help of Keynesian models, they were evidently consistent enough with Keynesian predictions that prominent Keynesian economists endorsed the forecasts.

Heck, there's nothing wrong with it. Those people made serious mistakes, or were at least seriously misleading the public. There is no doubt they can find some way to explain why these events do not actually contradict their theories, but they still made errors, out of one kind of ignorance or another. We can still bash them for hubris and arrogance -- they deserve it, but then so do all those people who have been predicting imminent hyperinflation for the last 3 years.

[Jim]

Steve is right to use this comparison as a meaningful measure of the success of the ARRA, because the May 2009 CEA "Estimates of Job Creation" report recommends exactly this procedure.

"One macroeconomic check will be to compare aggregate measures of employment at the reporting date to a number of baseline forecasts (without stimulus) done at the time the act was passed. This is a way to quantify whether employment is higher than it otherwise would have been, and thus provides empirical rigor to the jobs estimates at the time of passage."

[fundamentalist]

Daniel: "Now let's take this one step further - do we use this to conclude that we don't have theories that adequately explain weather? Of course not. Because there's a difference between empirical verification and forecasting."

I disagree. Most people connect the bad forecasts with bad theory, otherwise the forecasts would be better.

Daniel: "To improve forecasting - yes. To test theory - no."

Mainstream econ has chosen one way to test theory - by empirical verification of their models through forecasting. Mainstream econ made the decision to test their theories that way. I think we should respect their decision.

Austrians have an alternative for testing theory and don't place much importance on historical data. But mainstream econ has set the rules for their theory and I think we should respect them.

How would you test mainstream theory?

[Jerry O'Driscoll]

For what it is worth, I am just off a panel on Fox Business News on the Fed. The FOMC policy directive was released.

The panellists generally acknowledged that, whatever benefit might have accrued from past Fed policy, there is nothing more helpful it can do going forward to stimulate growth (policy ineffectiveness). One panellist called for the abolition of the Fed ("the arrogance" that a group of men can determine interest rates). Another said it was time the Fed started "getting it right" if it doesn't want to be abolished.

The failure of monetary policy complements the failure of fiscal policy (as documented by Steve Horwitz). As an article in today's WSJ details, the staff forecast the Fed Board has consistently overstimated the impact on growth of monetary policy.

We are witnessing the systemic failure of macro policy. As Mario has observed, it may be due to bad science, bad art, or bad ethics. But the policies have failed.

[Troy Camplin]

Lee,

No, he can't have it both ways. All you can do with a complex system or process is make pattern predictions. The kinds of predictions necessary for Keynesian or other kinds of interferences in the market process to work are literally impossible. Daniel wants to support Keynesian interferences and regulations in the economy on the one hand, then when people point out that they things are wrecking havoc turn around and say that they can't know what they are talking about because the economy is a complex system. But he has it backwards. To do what he supports, you have to be able to make accurate predictions of outcomes, while the critics are making pattern analyses and predictions.

[Daniel Kuehn]

Fundamentalist -

re: "I disagree. Most people connect the bad forecasts with bad theory, otherwise the forecasts would be better."

Correct me if I'm wrong, but the theory that produces the forecasts and the theory the produces the stimulus estimate are two entirely different theories. Click through von Pepe's link above. Producing a baseline forecast is essentially an econometric exercise. It has nothing whatsoever to do with the theory that justifies stimulus.

re: "Mainstream econ has chosen one way to test theory - by empirical verification of their models through forecasting. Mainstream econ made the decision to test their theories that way. I think we should respect their decision."

Who? Could you be more specific? Is this how Barro tests fiscal policy? Romer? WHO has "chosen" this? Jim made me blink - he provided one example. This has not been typical in my experience. Can you name me a single article from a major journal that tests fiscal stimulus by comparing it to an ex ante forecast? Who are you talking about here when you say that the mainstream has chosen?

re: "Austrians have an alternative for testing theory and don't place much importance on historical data. But mainstream econ has set the rules for their theory and I think we should respect them."

OK - but now you're talking about something completely different. Historical data? Yes. You just need to provide a convincing identification strategy. But ex ante forecast baselines? That's very different. You need to make up your mind - are you talking about historical data or are you talking about forecasted baselines?

[Daniel Kuehn]

re: "Daniel wants to support Keynesian interferences and regulations in the economy on the one hand, then when people point out that they things are wrecking havoc turn around and say that they can't know what they are talking about because the economy is a complex system."

No Troy, this is absolutely not what I've said.

I've said that people can't make reliable prospective forecasts, not that they can't make valid empirical critiques.

Barro's approach is valid if you want to talk about someone critical of stimulus.

The approach outlined here is invalid.

I have not said that "when people point out that they things are wrecking havoc turn around and say that they can't know what they are talking about because the economy is a complex system".

[Martin]

"I agree with Martin's criticism of Bob Murphy."

Euuhh, I was referring to the post made by Steve Horwitz. Bob Murphy, in general, does not preach to the choir but actually is quite, quite convincing to those outside of his own penumbra's. For example, as an atheist I find his 'Sunday posts' to be quite inspiring.

As for the comments afterwards: If Steve and Daniel could agree to a debate topic on this issue, I'd be very interested to see what either party has to say. Perhaps as a warm-up to the Krugman-Murphy debate... ;-)?

[Daniel Kuehn]

Martin -
The only expressed disagreements Steve and I seem to have here is:

1. Whether Romer and Bernstein ever expected ex ante forecasts to form the basis of an empirical assessment of the stimulus, and

2. Given an affirmative to the first question (Steve's position) whether answering that question is at all interesting.

I think we disagree on both of these, but I don't think it's very promising fodder for a debate.

Clearly I agree with Steve that the little red dots are above the blue lines.

[Jim]

If government stimulus 'works' then surely spending $1 trillion should have done 'something.'

That is, assuming the stimulus was not large enough, can we at least ask for evidence that we got anything but forward buying and government expenditure queuing from the almost $1 trillion we spent?

What IS the spin-off? Where is the multiplier? To be horribly practical about it, as is the practice of any business, "Show me the numbers, or shut up."

It is NOT ok to just say the stimulus was not large enough. Only in academics could such a silly thing be a valid rejoinder. It is a mind game of gob-smacking irresponsibility for the people who are caught in the stagflation caused by a government running cute little experiments with other people's money.

[Current]

Martin,

Ah, so Steve Horwitz said that inflation and interest rates would rise? He may have I can't remember. If he did then that's a legitimate criticism of his predictions, but it doesn't make the criticism of the predictions of others less valid.

[mobsrule]

What test do economists recommend to compare the validity of their paradigms with those of others'? In less politicized science, we'd choose to follow a method that could accurately predict the future, or we'd choose to follow a method that had proved better at predicting the future than other methods (current weather forecasting.)

Or, if someone like Romer or Kuehn claims that he should be allowed to direct a trillion dollars toward pyramid building with a specific employment goal in mind, then why should we think that there's ANY good reason to acquiesce?

[Troy Camplin]

Why propose to do something if you can't predict its result? Or at least think you can't?

It is relevant to point out that those who wanted Keynesian stimulus made a prediction about what would happen if there were a stimulus, and if there were not, and compare the actual results to what was claimed. It is also relevant -- and rational -- to also conclude that even more of what made things worse won't make things better.

It is also true that if the economy is a complex process, we cannot predict what the outcome of any intervention will be, and to claim otherwise is to have a pretense of knowledge. To on the one hand argue that the economy is too complex to make predictions and on the other hand that it's not so complex as to make interventions in that economy with predictable results is at best inconsistent.

[Daniel Kuehn]

Troy, you've lost me. Don't we do just that every day of our lives?

So let's take a complex system we've been talking about - the weather. Weather forecasts are never perfect. I'm never sure exactly when a storm starts and I'm never sure exactly how torrential it will be and I'm ever sure exactly how long it will last.

That doesn't mean it's illegitimate for me to (1.) bring an umbrella with me into work in anticipation of rain, and (2.) use the umbrella when it's raining! I don't know how long the rain will last or how hard it will be. I can't know that. There are simply too many variables and too many feedback loops. But we aren't flying completely blind - we have some decent information to work off of. And ultimately the value of our solution to the problem (umbrella protecting me from rain) is completely independent of our ability to forecast the extent of the problem.

That's the key point here, Troy. The theoretical and empirical work that goes into predicting the impact of the stimulus on the one hand and forecasting the macroeconomy are two completely different things. Just like your ability to know "the umbrella will protect me from the rain" and your ability to know "the rain will last four hours and drop an inch of rain" are two completely different sets of knowledge.

We are bad at forecasting weather. We are pretty good at keeping our heads dry. Unfortunately bringing down unemployment is a lot harder than opening an umbrella, but your conflation of the two issues at hand is just as wrong as in the umbrella example.

I don't have a pretense of knowledge. I'm not being inconsistent. You are conflating two entirely separate issues: the baseline forecast and the impact estimate. Trust me - I produce impact estimates for a living (not macroeconomic impacts, but the concept is the same). You are crossing your wires. Pick up a paper by Barro or by Romer and then compare it to a paper by Stock or by Watson. These are completely different questions.

[Steve Horwitz]

Martin,

Like Current, I thought you were referring to Bob, who HAS made those predictions and, to his credit, blogged about the fact that he was wrong and has tried to figure out why.

I think you'll look in vain for me making any prediction about high inflation or interest rates due to QE1 or QE2, at least in the short run. I have said that they risk moderate (e.g., 10%) rates of inflation 3-5 years out. I have also spent a lot of time explaining why they haven't been inflationary now, e.g., why they are sitting in bank reserves for the most part. I have shied away from making specific predictions for all the reasons in play on this thread.

So if you have me making a specific prediction about significant inflation in the direct aftermath of the QEs, please provide chapter and verse.

[Steve Horwitz]

Jim,

Thank you for that CEA quote, as that's exactly the point I've been trying to make and there is the CEA saying it for me.

[Daniel Kuehn]

Steve -
Jim's point was good, but:

1. It's worth finishing off the paragraph he quotes a portion of.

2. I'm still not clear who actually does this to assess fiscal policy. None of the work I'm familiar with (and again, I haven't been in the business as long as you guys) does this. There's probably a good reason for that, since doing this is tremendously easier than what is done.

[fundamentalist]

Daniel: “Producing a baseline forecast is essentially an econometric exercise. It has nothing whatsoever to do with the theory that justifies stimulus.”
You lost me there. How can you build an econometric model without some kind of theory? The Blinder model incorporated Keynesian theory that justifies stimuli and then predicted that it would work. That’s the problem Kling was pointing out.
Daniel: “Is this how Barro tests fiscal policy? Romer?”
I would imagine they do. You have read the few mainstream economists who do methodology. And when Krugman, et al, discuss methodology they make it clear as to what they would consider proof of a theory. Do you know any mainstream economists who have rejected positivism?
Daniel: “are you talking about historical data or are you talking about forecasted baselines?”
You fail to mention what Kling added to the end of “"baseline scenario": “that matches history and then projects out to the future.” It’s really difficult to forecast without a model, and even harder to make a model without historical data. They all kinda go together.

[fundamentalist]

Jim: "It is NOT ok to just say the stimulus was not large enough."

Excellent point! Mainstream economists insist that the stimuli simply weren't large enough, as if they know for a fact that stimuli and monetary pumping act like neurons and have a stimulus threshold to cross before the economy fires. I have asked several mainstream economists to show me proof that a threshold exists.

[Troy Camplin]

Daniel, if someone predicted the weather like Keynesians predict the economy, they would have been fired and never worked in meteorology again. ANd nobody is so idiotic as to believe they can control the rain.

[Jon O.]

This only proves the administration's forecast was incorrect. We can criticize them for this but not the stimulus qua stimulus(we can, of course, criticize the stimulus in other ways.) I don't think it's fair to hold all pro-stimulus types - and the stimulus itself - to the administratoin's forecast; just as it's not fair to hold all stimulus opponents to certain austrian forecasts on rates and inflation.

In the late 08' - early 09' period I had a number of discussions with colleagues that thought if there was no large fiscal expansion we'd have GD-like unemployment and deflation within 2-yrs. I guess they were correct and the stimulus succeeded?

The point is: there were a number of forecasts about what would happen with and without stimulus, none of which proves anything - except maybe who the worst forecasters are.

[McKinney]

I would be interested to hear from Daniel how he thinks economists should test theories.

[Daniel Kuehn]

fundamentalist -

re: "You lost me there. How can you build an econometric model without some kind of theory? The Blinder model incorporated Keynesian theory that justifies stimuli and then predicted that it would work. That’s the problem Kling was pointing out."

So forecasting models are fancy versions of estimating y(t)=y(t-1). Add in various filters, lag structures, covariates, etc. etc. and you get your forecasts. It has nothing at all to do with the Keynesian theory of fiscal policy. Let me put it to you this way - how would you even forecast with a standard Keynesian model? That's not what it's designed to do. The world that Barro and Romer live in - the world that produces estimates of the impact of the stimulus - is a completely different world and literature from that which Stock and Watson live in.

If you read Kling's post it's clear that these are two separate estimates - steps 1 and 2 are Stock and Watson's world. Steps 3 through 6 are Romer and Barro's world. Kling knows what he's talking about.

[Daniel Kuehn]

fundamentalist -

There are two very different empirical things going on here. There are a group of people that try to identify:

y = a + bG

to get an estimate of b. Identification of this estimate is very hard. Then there are a group of people who try to estimate:

y(t) = a(y(t-1))

And use it to say something about y(t+1).

These are two very different tasks that you are trying to pretend are the same thing and that you are trying to pretend are derived from the same theory.

The second group of people were off. But we've known for a while that estimating the second kind of model is very hard to get right too far into the future.

That has no bearing on the estimates produced by the first group of people.

[Daniel Kuehn]

Perhaps I should say this more clearly - the consistency and the efficiency of the coefficient on G in the first case and on y(t-1) in the second case have no bearing on each other, and if we think the coefficients on y(t-1) are particularly dicey (we do think this) it's an awfully odd choice for testing the veracity of our estimated coefficients for G.

[McKinney]

I understand what you're saying, but my position is much simpler: Why did you choose G as your predictor in y=a+bG?

You must have some theory that says G causes Y.

BTW, read other posts by Kling on such modeling and you'll see he has a great deal of contempt for that process. He has written that the process does little more than assume the conclusions.

[Martin]

I see that my earlier post, has not gotten through (probably my internet connection acting up).

@Steven: I had a brief look through your posts, but I could not find it. I stand corrected, my apologies.

[Silvano Fait]

@Daniel Kuehn

They told people need a plan.
Then they told: "we have the right one".
Then they enacted the plan promptly.

Now the relevant question is: Are we facing or not a Government failure?
May we trust again these planners as they are (and not as we would like they were) again or not?

An "aprioristic" defence of all the Keynesian universe is useless. I mean, it's not required. It's out of the scope of the post.

[TJL]

>That [May] 3.57% annual rate is higher than the average annual inflation rate (measured by the CPI) in every year from 2000 to 2007

Seeing as month-to-month rate is quite volatile compared to the annual rate, I find it fairly disingenuous to write the above line. We've seen rising inflation, but there have been plenty of months in the 2000-2007 period that exceeded 3.57%.

[Steve Horwitz]

Okay TJL, how about we try it this way then?

Between Jan 2000 and May 2011, the annual rate of inflation was greater than 3.57% (the May 11 value) for 27 of those 137 months, or about 20%. Of those 27 months when annual inflation was higher, 10 of them (or nearly half) were in 2008, the one year in which the annual rate for the year was higher than that of May 2011.

[Russnelson]

I hesitate to respond to Daniel, because he ultimately a poopy-head, however, I'll point out that Keynesians made a forecast that failed. We don't really care what Austrians think, because Austrians didn't make that forecast (because as Daniel points out, that's a poopy-head thing to do). However, the Keynesians made the forecast and used it as part of the reasoning for their public policy. Now that the forecast has been seen to be false, so must also be the public policy based on it. Thus, the public policy should be reversed.

If you want to make me feel better about you, Daniel, point out a time when stimulus has worked better than leaving the economy alone AND plausibly committing to do so.

I'm waiting ... but I'm not going to hold my breath.