policy-relevant research

Steve Walt weighs in with his take on the relative virtues of NSF funding of political science.  I agree with a fair amount of what he wrote (in particular the lNSF's listing of sponsored research outputs), but this part brought me up short: 

I can't say that I think Coburn is right, but I'm finding it hard to get too exercised about it. I say this in part because I think a lot of NSF-funded research has contributed to the "cult of irrelevance" that infects a lot of political science, and because the definition of "science" that has guided the grant-making process is excessively narrow.  But I also worry that trying to use federal dollars to encourage more policy-relevant research would end up politicizing academic life in some unfortunate ways.

Walt is conflating two different things here -- "policy-relevant research" and "publicly beneficial research."  Believe it or not, those two terms are not equivalent. 

The implicit assumption in Walt's post -- and a lot of discussions on this topic -- is that if political science research cannot produce policy-relevant advice, then it's not worthy of public funding.  But this gets the argument exactly backwards.  One would assume that, the greater the demand is for policy-relevant research, the more outsourcing and consultancies that would be pursued.  And, indeed, I think that's what you're seeing with the rise of political risk consultancies and the Defense Department's Minerva project. 

The key question to ask is whether that kind of policy-relevant research can be produced out of whole cloth or whether it rests on more basic research into political science and international relations -- the kind of basic research for which the free market would underprovide.  Much of Walt's own research, for example, rests on Kenneth Waltz's Theory of International Politics.  This is a book that proffers very little in the way of useful policy advice.  It is, nevertheless, a foundational text; an awful lot of realists build their policy prescriptions off of that book (and, if memory serves, Waltz received NSF funding to write that book).  Speaking for myself, a lot of what I wrote in All Politics Is Global is cribbed from rests on Albert Hirschman's more abstract work Exit, Voice and Loyalty. 

There is a continuum of research that exists in the socal sciences.  One could start with basic theoretical work and empirical data collection that seems far removed from policy relevance, and move to finely detailed policy memoranda.  I don't think the latter are terribly useful without resting on the former -- and one could argue that it's the former that would be underprovided without NSF funding. 

But I could very well be wrong -- perhaps policy analysis can be done independently of more abstract theories and models of political science.  That's a discussion worth having.  Requiring NSF-funded projects to have immediate policy relevance, however, cedes way too much terrain to critics of the discipline.  As Nobel-Prize-winning Elinor Ostrom pointed out, sometimes it's worth investigating the seemingly obvious -- because sometimes the obvious is wrong. 

Following up on the Tom Coburn saga, Patricia Cohen has a round-up in the New York Times about whether the study of political science contributes to the public good.  Some excerpts: 

Much of the political science work financed by the National Science Foundation is both rigorous and valuable, said Jeffrey C. Isaac, a professor at Indiana University in Bloomington, where one new winner of the Nobel in economic science, the political scientist Elinor Ostrom, teaches. “But we’re kidding ourselves if we think this research typically has the obvious public benefit we claim for it,” he said. “We political scientists can and should do a better job of making the public relevance of our work clearer and of doing more relevant work.”

Mr. Isaac is the editor of Perspectives on Politics, a journal that was created by the field’s professional organization to bridge the divide after a group of political scientists led a revolt against the growing influence of statistical methods and mathematics-based models in the discipline. In 2000 an anonymous political scientist who called himself Mr. Perestroika roused scores of colleagues to protest the organization, the American Political Science Association, and its flagship journal, The American Political Science Review, arguing that the two were marginalizing scholars who focused on traditional research based on history, culture and archives.

Though there is still jockeying over jobs, power and prestige — particularly in an era of shrinking budgets — much of that animus has quieted, and most political scientists agree that a wide range of approaches makes sense.

What remains, though, is a nagging concern that the field is not producing work that matters. “The danger is that political science is moving in the direction of saying more and more about less and less,” said Joseph Nye, a professor at the John F. Kennedy School of Government at Harvard, whose work has been particularly influential among American policy makers. “There are parts of the academy which, in the effort to be scientific, feel we should stay away from policy,” Mr. Nye said, that “it interferes with the science.”

In his view statistical techniques too often determine what kind of research political scientists do, pushing them further into narrow specializations cut off from real-world concerns. The motivation to be precise, Mr. Nye warned, has overtaken the impulse to be relevant.

[Full disclosure:  I'm not now on the editorial board of Perspectives on Politics, and therefore am obligated to link to Isaac's Chronicle of Higher Education essay on this topic.] 

Coburn's focus has been on the past ten years, and I think the biggest irony of that focus is that, compared to a decade ago, there's more policy-relevant research and less paradigmatic navel-gazing. 

[Got any hard evidence, smart guy?--ed.]  This is very tough to measure (if only we had an NSF grant!), but consider the following:

• A political scientist just won the Nobel Prize in Economics using a cluster of different methodologies;
• Demand for the services of political scientists has never been greater.  The Defense Department has awarded political scientists through its Minerva program.  Political risk consultants are all the rage now -- and while I won't fully endorse their product, the fact that firms feel like they are worth paying gobs of money is an example of revealed preference;
• In the past year, political scientists have influenced the public debate on issues such as the Bradley effect in elections or U.S. standing in the world.
• Whether you agree or disagree with them, one can point to several episodes in recent years in which political scientists have provoked some roiling debates over a public policy issue. 
• This is an insufficient sample, but I'd note that political scientists are better represented in the most recent issue of Foreign Affairs than they were in the issue from a decade ago.  Assignment to aspiring researchers/bloggers:  go through the archives of Foreign Affairs, Foreign Policy, The National Interest, Policy Review, and The Washington Quarterly and see if there are more political scientists publishing in these journals recently than, say, a decade ago. 

I'm planning on posting why I think political sciece is in better shape than it was a decade ago later in the week.  But for now, a question to readers:  are these examples persuasive, or do you need to see more evidence? 

Matt Yglesias linked to this months-old Emily Stokes profile of Rory Stewart in the Financial Times.  Yglesias highlights one of the funnier metaphors I've seen about the trouble with advising policymakers:   

“It’s like they’re coming in and saying to you, ‘I’m going to drive my car off a cliff. Should I or should I not wear a seatbelt?’ And you say, ‘I don’t think you should drive your car off the cliff.’ And they say, ‘No, no, that bit’s already been decided – the question is whether to wear a seatbelt.’ And you say, ‘Well, you might as well wear a seatbelt.’ And then they say, ‘We’ve consulted with policy expert Rory Stewart and he says ...’"

OK, that's really funny, and I think it's true a fair amount of the time. 

On the other hand, I'm not sure that metaphor holds up all of the time.  Consider another possibility.  From the policymaker's perspective, getting outside advice is like trying to figure out which railroad track to take if you're driving a train.  There are three options ahead, and for myriad reasons each of the possibilities carries some risk.  So you go place an emergency phone call to the head of Harvard's Department of Railroad Studies to get a recommendation.  His advice?  "Why don't you go off-track?"  

To revisit a recurring theme on this blog, sometimes the outside advisor is right to make policymakers question core assumptions.  At the same time, however, sometimes a policymaker has neither the time nor the political capital to go back to first principles.  Sometimes they just need to know what is the least bad policy option.  And I guarantee you that having an academic tell them, "they're all bad policy options" is of no use whatsoever in that moment.   

I suspect that knowing which metaphor applies is more art than science, but I'm curious to hear from commenters on both sides of the policymaking divide. 

A few days ago, Nick Pope had an unusual op-ed in the New York Times.  It was about unidentified flying objects and why the United States should take them more seriously: 

A healthy skepticism about extraterrestrial space travelers leads people to disregard U.F.O. sightings without a moment’s thought. But in the United States, this translates into overdependence on radar data and indifference to all kinds of unidentified aircraft — a weakness that could be exploited by terrorists or anyone seeking to engage in espionage against the United States. The American government has not investigated U.F.O. sightings since 1969, when the Air Force ended Project Blue Book, an effort to scientifically analyze all sightings to see if any posed a threat to national security. Britain and France, in contrast, continue to investigate U.F.O. sightings, because of concerns that some sightings might be attributable to foreign military aircraft breaching their airspace, or to foreign space-based systems of interest to the intelligence community.... The United States is no less vulnerable than Britain and France to threats to security and air safety. The United States Air Force or the National Aeronautics and Space Administration should reopen investigations of U.F.O. phenomena. It would not imply that the country has suddenly started believing in little green men. It would simply recognize the possibility that radar alone cannot always tell us what’s out there.

Why, why indeed has the United States been so reluctant to, "recognize the possibility that radar alone cannot always tell us what’s out there"?  Oddly enough, international relations theory has an answer.  I'm not saying it's the right answer, mind you, but it's a very interesting answer.  My dear readers, I give you Alexander Wendt and Raymond Duvall, "Sovereignty and the UFO," Political Theory, Volume 36, Number 4 (August 2008):  607-633.  Here's the abstract: 

Modern sovereignty is anthropocentric, constituted and organized by reference to human beings alone. Although a metaphysical assumption, anthropocentrism is of immense practical import, enabling modern states to command loyalty and resources from their subjects in pursuit of political projects. It has limits, however, which are brought clearly into view by the authoritative taboo on taking UFOs seriously. UFOs have never been systematically investigated by science or the state, because it is assumed to be known that none are extraterrestrial. Yet in fact this is not known, which makes the UFO taboo puzzling given the ET possibility. Drawing on the work of Giorgio Agamben, Michel Foucault, and Jacques Derrida, the puzzle is explained by the functional imperatives of anthropocentric sovereignty, which cannot decide a UFO exception to anthropocentrism while preserving the ability to make such a decision. The UFO can be "known" only by not asking what it is. 

Now I suspect that approximately 95% of my readers are going to dismiss Duvall and Wendt's argument just based on the abstract.*  Indeed, the abstract as written seems consciously designed to go straight into an amusing post on NRO's The Corner [That's right, he's daring you!!--ed.]   Hell, I've read the paper and my immediate reaction to that abstract was similar to Michael Munger's.    That would be a mistake, however.  Wendt and Duvall are quite careful to distinguish between the literal category of "unidentified flying objects" and the hypothesis that those unidedentified flying objects are actually little green men.  They then attempt to develop a theory for why Pope's warning has not been heeded.  That said, I highly recommend Henry Farrell's analysis of the paper to see the flaws in their argument.  Still, give Wendt and Duvall some credit -- it's pretty rare to write about a topic this arcane and then find a New York Times op-ed on the same subject come out the same week as a journal paper.  *In my experience, a big problem with abstracts is that they are often the last thing written, hastily cribbed together in order to get a paper off one's own desk and into a publication.  This is a shame, since a paper's abstract is likely to be read far more often than the paper itself.