Tuesday, March 25, 2014

What if everything we know is wrong?

I have been reading a few things lately, and a horrifying idea began to coalesce in my head. Let me go over the literature so you can get an idea of where I’m coming from.

Americans are the weirdest people in the world: This provocatively titled paper is a cross-cultural psychological study, that shows people from different backgrounds having drastically different results to psychological tests. This might sound intuitive at first, but their results challenge some of what psychologists have maintain are UNIVERSAL psychological principles, from how we understand social relations to even how we interpret what we see with our eyes. Without going into too much detail; the paper shows that, of all people in the world, Western people tend to be outliers on many psychological tests, and Americans are outliers even among that group. We see and experience the world in a very different way than the majority of humanity. The authors believe we need to really rethink our bedrock psychological assumptions.

Paul Krugman made a few blog posts going after Nate Silver’s new website, in end critiquing the use of generic data analysis and economics tools to look at problems that those tools were not really designed for and lacking real expertise in the fields he is analyzing. He is a little afraid that what will come out of this “is if this turns into a Freakonomics-type exercise, all contrarianism without any appreciation for the importance of actual expertise.” He links to a Michael Mann’s review of Silver’s book, on which he is strongly disappointed with Silver’s application of economic analysis to the problem of climate change. I’ll quote for clarity: “[Silver] falls victim to a fallacy that has become all too common among those who view the issue through the prism of economics rather than science. Nate conflates problems of prediction in the realm of human behavior -- where there are no fundamental governing 'laws' and any "predictions" are potentially laden with subjective and untestable assumptions -- with problems such as climate change, which are governed by laws of physics, like the greenhouse effect, that are true whether or not you choose to believe them.” This seems to be a running problem for economists, in fact: Mann also discusses “University of Chicago economist Steven Levitt, known largely for his provocative 2005 book Freakonomics and its even more audacious 2009 sequel Super Freakonomics -- a book that, perhaps better than any other, serves as a cautionary tale of the dangers that lurk when academics attempt to draw sweeping conclusions in fields well outside their area of training.” Basically, these critical analysis have the combined theme of the dangers of applying the methods and theories of one field to another, or of letting “data speak for itself”, which is often worse since, as Krugman says, “If you think the data are speaking for themselves, what you’re really doing is implicit theorizing, which is a really bad idea (because you can’t test your assumptions if you don’t even know what you’re assuming.)”

In the most recent issue (March 2014) of Foreign Affairs, an article was written that attacked our use of “leading indicators” to do economic analysis. Each of the major indicators is flawed in a major way, he argues, leaving out information that the original designers thought wasn’t important or would be too difficult to calculate. And then these numbers are viewed through the lens of expectations. Growth that would be high for one country (let's say 6% GDP growth in the US) would be considered quite low for another (China). Karabell puts it pretty well: "A GDP that is growing in sync with expectations can enhance a country’s reputation and thus its strength and power. A GDP that is contracting or failing to meet expectations, on the other hand, can lead to disaster." Others take these numbers (GDP, inflation rate, balance of trade), which are already flawed, and use them to make analysis of their own industry or as proxies for other measurements (like using GDP and population as a measure of state latent power).  Let me give a very nerdy example that I am sure of the facts on (this is basically an extreme version of the current situation in Saudi Arabia, but I don't know the case well enough and I'm sure it's more complicated than what I am about to present to you). In the Star Trek universe, Ferangi women are forbidden by law from earning the profit, or generally even leaving the home. They are completely unproductive members of society by any measure of GDP, since they can’t work. How does that affect the power of the Ferangi alliance? If one looked at the size of its workforce and army, or total production, it might look much lower than that of the Federation, or the Romulans. But if we translate that into the latent power of the Ferangi, we would be hugely off the mark. At any time, they could free their women, effectively doubling their workforce and potential military size. But that fact is not present in the numbers. The key point here is that our data is distorting our view of reality, and models based on it will, at some point, always be wrong. And the farther removed you get from the original point of the data (using indicators to gauge overall economic health, using economic health to gauge overall state power) the most distorted things will be.

Take these three points, and then consider the following: it is well known that social scientists fetishize economists, because they appear to be the most “science-y” of the group. Economists themselves fetishize the hard sciences, because they can actually follow the scientific method in laboratory experiments, have unbreakable laws of nature, etc. All science wants to be biology or physics, or optimally, pure math. This is a huge fallacy, the fact that social sciences are SOCIAL makes applying lessons from the very unsocial hard sciences (there is no society of stars or atoms or cells in a human body) to a social science like economics difficult at best. Worse yet is take what we have learned about economics (to the extent that we have learned anything) and apply it to other fields. Just today I was reading a piece about conspicuous consumption in international relations and how relative gains considerations were extremely important when determining consumption, and the following line came up “Hence, we should expect consumption patterns that often do not follow economic predictions of maximizing utility while minimizing costs. Consequently, we should be very cautious when applying economic models, which rely on the theory of demand, to the study of international relations.”

International Relations, as an academic discipline, is a fairly new field, developed mostly in America in the years following World War II. It was barely established when the field was rocked by the movement towards more rational, “scientific” methods in the 70s and 80s. Waltz’s famous Theory of International Relations (1979) is basically an extended microeconomics analogy. Econometric analysis and game theoretic formal models are everywhere, and they all make the rational choice assumptions that come from econ: states behavior rationally, make cost benefit analyses and act accordingly, etc. I realize these are models and all models make assumptions, but are these the right assumptions? Or are we just borrowing from economics?  

To make matters worse, almost all of the major IR scholars in the field are from Western Europe or America, and mostly America. Check out this list:

Kenneth Waltz, born in Ann Arbor
John Mearsheimer, born in Brooklyn
Robert Keohane, born in Chicago
Joseph Nye, born in New Jersey
Alexander Wendt, born in West Germany, educated in America
Francis Fukuyama, born in Chicago
Henry Kissinger, born in Germany, educated in America
Samuel Huntington, born in New York
Ernst Haas, born in Germany, educated in America

The list goes on, but basically nearly all scholars are western, mostly American, and those that aren’t American born were educated here (and were apparently all in Germany before that). Now, take this information and connect it to the first piece I discussed above. Being American gives you, apparently, a very unique view of the world. Nearly all of modern international relations comes from Americans. How does this monolithic american perspective affect IR theory? What American biases and unstated assumptions (assumptions that the author doesn’t even know they have because the assumption is ‘supposed’ to be universally applicable!) underlie all the work that has been done?

So, let’s take this all together. International Relations is built out of a primarily American academic base, which is an extreme outlier of the general population by many measures. It was also developed during a unique time in world history, there has never really been a situation like the Cold War before. Any or all seminal International Relations scholarship might be unique to the American context or world view and make no sense to someone from another culture. Most of the tools used to study politics and international relations are adapted from economics and may or may not actually be appropriate for studying politics in this way. Our methods are built to emulate the work done in the hard sciences rather than being customized to the needs of the field.

What if everything we know is wrong?

This is an absurd list of possible research problems. How can we expect to really gain explanatory power when we have crazy distortions and biases underlying both the theory AND empirics? We have no idea if our theories make sense, and our lack of true understanding of the data means even if our theory was good and it SEEMED to fit the data, it still might not be true in reality because our empirical data was distorted! People wonder why political science seems to have such crappy theories. Mearsheimer always says that if a theory can explain even 2/3s of cases, it’s pretty great by political science standards…could this be because we are using bad tools, bad methods, and our field is skewed towards a culture that no one else in the world shares? These fundamental issues need to be solved or our field is never going to achieve anything close to a universal ‘law of politics’, like the laws that exist in physics or chemistry or even biology.

1 comment:

  1. What would political science look like without these research problems? How can we address them? I guess we need more cultural perspectives in political science. I guess political science needs to be respected as a field of its own separate from physical science, based in human behavior and culture rather than absolute laws. Any political science research you find especially impressive?

    ReplyDelete