Over at EconLog, my dear friend and colleague Bryan Caplan is channeling Alfred Marshall's (Burn the Mathematics) and Murray Rothbard's (Occam's razor) critiques of the overuse and misuse of mathematical modeling in the discipline of economics. I am no doubt biased, but all my intellectual sympathies are with Caplan on this issue. And I believe his proposed test actually is a very good one.
To remind the young aspiring economists readers of the arguments, Marshall in 1906 wrote that he had "a growing feeling in the later years of my work at the subject that a good mathematical theorem dealing with economic hypotheses was very unlikely to be good economics: and I went more and more on the rules - (1) Use mathematics as a shorthand language, rather than an engine of inquiry. (2) Keep to them till you have done. (3) Translate into English. (4) Then illustrate by examples that are important in real life. (5) Burn the mathematics. (6) If you can't succeed in (4), burn (3). This last I did often."
And Murray Rothbard, at least in part, argued in Man, Economy, and State that the dance of thinking through the economic intuition, translating it into math, then translating it back into English to explain was a violation of the philosophical and scientific principle of Occam's razor. But Rothbard did go further in his criticism, as he put it: "Aside from doing no more than verbal logic can do, and therefore violating the scientific principle of Occam’s razor—that science should be as simple and clear as possible—such a use of mathematics contains grave errors and defects within itself. In the first place, it cannot describe the path by which the economy approaches the final equilibrium position. This task can be performed only by verbal, logical analysis of the causal action of human beings. It is evident that this task is the important one, since it is this analysis that is significant for human action. Action moves along a path and is not describable in an unchanging, evenly rotating world. The world is an uncertain one, and we shall see shortly that we cannot even pursue to its logical conclusion the analysis of a static, evenly rotating economy. The assumption of an evenly rotating economy is only an auxiliary tool in aiding us in the analysis of real action. Since mathematics is least badly accommodated to a static state, mathematical writers have tended to be preoccupied with this state, thus providing a particularly misleading picture of the world of action."
Economics, mid-20th century, took a turn toward excessive formalism and excessive aggregation, and as a discipline it hasn't fully recovered yet. In many ways, common-sense economics was a casualty of this turn. But one of the great conscientious objectors to the excessive formalization of economics was Kenneth Boulding. Boulding was the 2nd winner of the John Bates Clark Medal from the American Economic Association in 1949 (Samuelson was the 1st winner in 1947). Boulding reviewed Samuelson's Foundations in the Journal of Political Economy in 1948, and as he put it: "Conventions of generality and mathematical elegance may be just as much barriers to the attainment and diffusion of knowledge as may contentment with particularity and literary vagueness.... It may well be that the slovenly and literary borderland between economics and sociology will be the most fruitful building ground during the years to come and that mathematical economics will remain too flawless in its perfection to be very fruitful".
Mathematical reasoning can be a very useful servant to economic theorizing, but it is a horrible master. Economics has suffered from a slavish devotion to mathematical modeling and statistical testing for over 60 years since Boulding wrote he prophetic warning. I hope the young aspiring economists reading this, will take motivation from Bryan Caplan's posts and study the matter seriously. Thinking through the arguments made on both sides, study not only contemporary pronouncements, but trace back the arguments in time to the classic statements by Marshall, Mises, Hayek, Buchanan, Coase, Friedman, Rothbard, Lachmann, and of course Kenneth Boulding.
BTW, Deirdre McCloskey has a wonderful piece on Boulding that you can consult as well.