Does Google Scholar Capture Influence? And other perplexing questions

As reported, Pete Leeson and I are in London for a conference at the LSE.  As we were walking around yesterday we got in a discussion about who were the top economics since 1975 whose ideas have had an influence yet who were not published in the AER, JPE or QJE.  We were discounting papers and proceedings of the AEA, so papers in the AER from the meetings were not counted.  Instead, you had to think of economists who were unable to place their papers in the top journals due to an intellectually conservative bias in the refereeing process yet were able to influence the intellectual culture of economsts down the road. Buchanan, Coase, Kirzner, North, and Olson are some of the names we came up with.  We will have to investigate further and also check our premises on the names already listed.  Who would you nominate?

The question started when Pete asked me a question about my early career at NYU, and I told him about my experience in comparative and Sovietology.  Basically I argued that I decided to write primarily books in that field because I was questioning too many conventional assumptions at the same time for journal articles to work out.  When writing a journal article you are usually permitted to question 1 or perhaps 2 conventional assumptions at a time, but if you question too many I argued you cannot get through the journal refereeing process.  This is why I contend that Austrian economics remained a "book culture" through the 1980s and 1990s when the rest of the economics profession had become primarily a "journal culture" by 1950.  On the other hand, in the 1990s journals in the field of history of economic thought and methodlogy provided outlets for work in those fields and I did publish in HOPE, JHET, JEM, etc.

Early this morning I looked up Google Scholar to see if it accuratedly reflected influence.  Douglass North's Institutions, Institutional Change and Economic Performance comes up with over 5000 citations; James Buchanan's Calculus of Consent has over 900, but his other books get in the 500 to 900 range; Mancur Olson's Logic of Collective Action received over 2000 citations; Ronald Coase's "The Problem of Social Costs" has over 3000 citations; and Israel Kirzner's Competition and Entrepreneurship has 841.

Now how do these guys compare?  Paul Samuelson -- 950 for his "Pure Theory of Public Expenditure"; Robert Solow -- 2849 for his paper on growth; Ken Arrow -- 2548 for Social Choice and Individual Values.  F. A. Hayek gets 1647 for "The Use of Knowledge" and 800 for The Constitution of Liberty.  Miton Friedman gets 1496 for Capitalism and Freedom and 1046 for "The Role of Monetary Policy."  John Maynard Keynes's General Theory comes in at 2753. Adam Smith's Wealth of Nations only picks up 1978 citations, and Karl Marx's Capital comes in with 875.  Andrei Shleifer (the most cited economist in the past decade), however, comes in with over 2000 citations for his paper on "Law and Finance" but also has several other papers earning close to, or more than 1000 citations per paper.

Compare this with paltry numbers of 50-60 citations that are earned by my books on the history and collapse of the Soviet system.  My citation numbers are in line with Cowen, Garrison and Hoppe, White and Selgin do better in the field of free banking, and Caldwell does well in the field of history of thought and methodology, but Rizzo does worse in the field of law and economics as is the case with Salerno and Block in their respective fields.  Lavoie's Rivalry and Central Planning, Caldwell's Beyond Positivism and White's Free Banking in Britain are the most cited over 100 citations each. But it is obvious that our generation of Austrians is falling way behind in the race for scholarly influence even as measured on the web.  The best and most influential writers get in the 100-150 range, but not above for any one work.  We should aspire to be as influential as Kirzner or Hayek.  Even Mises, who many think is ignored still received over 300 citations for Human Action and over 100 citations to his "Economic Calculation in the Socialist Commonwealth."

Obviously, topic and placement of the original article or book matters greatly for subsequent influence.  With the increase in the number of potential outlets it is probably important to stress to young scholars to not settle for easy outlets but to keep bucking the refereeing system and work hard to place articles in as highly recognized outlets as possible.  Otherwise, people may continue to have little to no impact despite publishing.  In addition, one should utilize the tools emerging on the web to highlight papers and books that find a physical published outlet in a less prestigious outlet.  As Tyler Cowen has stressed --- the internet is changing the entire certification relationship between author and publisher.  One most always ask the question who is certifying who -- if it is the author who is lending his/her name to the journal or publisher, or is it the publisher who is certifying the author.  With blogs, let alone SSRN and other scientific advertisement of research now available, perhaps even obscure outlets can still reach a wide audience.  But in the end I am reminded of an incident at NYU during my 8 year stay.  A junior faculty was going to earn tenure --- only 2 were granted tenure during my time there out of 10 candidates.  During the debate over the successful tenure bid a letter was written by a major economist that said the following (paraphrasing since I cannot quote exactly): "The candidates work in economic theory is among the best of his cohort.  Only about 5 people in the profession read his work, but those 5 really matter."  I remember myself thinking at that time that there was something wrong. But now I appreciate the point.  The internet might expand readership, but unless your work is reaching certain people and more importantly certain TYPES of people who are in turn using your work to advance their work (i.e., you become a productive input into the scholarly production function of others) then don't expect scholarly influence.

Use all the tools at your disposal to promote your work, but remember that your first goal should be to do high quality work that you can place in as high profile a scientific outlet as possible.  If you succeed in doing that, the promoting of your work across a wide body of readers will follow.

Comments

[Ott Lake Rambler]

I agree with the trend, Pete, and that underscores a more obvious tendency to engage in self-promotion. But I myself must be an outlier here, because I'm uncomfortable playing the self-promotion game.

Your 50-60 citations are not really "poultry" numbers -- I think -- but perhaps they are paltry. At the same time, those aggregate statistics can be misleading. After all, in your case, half of those citations come from you yourself!

[Pete]

Dave,

Thanks for the catch ... though the basic bottom line is same -- chicken **** contributions. And all the figures give self-citations, though obviously when you are getting over 100 the impact of your own work on the numbers is small.

I stand by my initial point made through 15 minutes of searching on line (which means I could be wildly off mark) that we are not making much of an impact in the scholarly world even as measured by the internet, and that compared to even Hayek and Kirzner, and way off the standard set by Buchanan, Coase, North and Olson, let alone Shleifer. We want to aspire to influence the scientific program of economists like Shleifer does, and not like we do at the moment.

Pete

[J Hall]

Pete,

I don't think Buchanan meets your test of someone who was influential but did not publish in the top journals after 1975. Look at Buchanan's vita.

"Polluters' Profits and Political Response: Direct Controls versus Taxes" (with G.Tullock), American Economic Review 65(March 1975): 139-47.

"A Contractarian Paradigm for Applying Economic Theory," American Economic Review 65(May 1975): 225-30.

"Markets, States, and the Extent of Morals," American Economic Review 68(May 1978): 364-68.

"The Homogenization of Heterogeneous Inputs" (with Robert Tollison), American Economic Review 71(March 1981): 28-38. As "The Homogenization of Heterogeneous Inputs: Reply," American Economic Review 74(September 1984): 808.

"The Constitution of Economic Policy," American Economic Review 77(June 1987): 243-50. (Nobel Lecture)

"Secession and the Limits of Taxation: Towards a Theory of Internal Exit" (with Roger Faith), American Economic Review 77(No. 5, December 1987): 1023-31.

Now his influence may have come about because of his earlier work, but it is not at all clear that he had difficulty publishing in the AER (or even other top journals) well into his mid 60s. And how many economists continue to publish in the AER after that age?

[carl marks]

I have a theory which might partially explain the small number of citations, especially for someone like Block. When refs see that you are citing people considered outside the norm your paper may be discredited, even if the point the person is making is non-controversial. Because of this, authors may choose to cite another source if one is available, or they may get rejected because of citing a controversial figure.

This obviously might not be the only reason, but it may have a long-term impact on the austrian movement as they are not only shut out of the major journals, but even papers that cite them may be shut out, giving them even less name recognition.

[Wendy]

I recently attended a seminar where journalists were taught to use Google Scholar to find the most influential thinkers on various subjects, so the lack of citations among Austrian economists might also reduce the exposure of those ideas to the general public as well.

[Rafe Champion]

The problem with being a book culture is that it is getting harder to persuade students to read books, especially if they are not going to be examined on the contents. I get the impression that many students think if it is not on the net, it doesn't exist. So we need to master cyberspace, if only to draw attention to the books.

[Pedro Romero]

Pete,
I really think this advice will shape my future career. Both as an economist and a follower of the austrian school. I will recall it every time I make a decision.