Blogging will be light for the rest of the week, as I'll be attending the International Studies Association annual meeting in San Francisco. 

If you're also attending but new to these things and therefore unsure of what the informal norms are about such events, check out Megan MacKenzie's indispensable ISA Guide to Newbie Graduate Students. Oh, and come attend the First Ever Official ISA Blogging Reception. I'll be there too, and I'm bringing my #TFC12 finalist flask with me!! 

My other piece of advice would be to read Rob Farley's provocative new PS: Political Science and Politics essay, "Complicating the Political Scientist as Blogger." Farley is responding to a 2011 essay by John Sides at the Monkey Cage, which offers what I would label the "standard" narrative about how blogging can be a help rather than a hindrance to good political science -- hell, I wrote something similar to it in 2008. 

Farley considers this standard narrative, ponders it for a second, and then puts all his chips into the middle and raises the stakes: 

Although I appreciate the effort to “just add blogging” to the discipline of political science, I worry that in making blogging safe, Sides gives away too much of what makes it interesting, influential, and fun. Specifically, I have two major objections to Sides’ characterization of blogging in political science. First, the article heralds an effort to discipline the political science blogosphere, establishing metrics for differentiating between “good” blogs that can contribute to (or at least should not be held against) a political science career, and “bad” blogs that do no one any good. In short, Sides’s article served both prescriptive and proscriptive purposes. Second, by emphasizing the “safe” elements of blogging, Sides has left winnings on the table; blogging could play a larger role in political science than he suggests.

Read the whole thing. I have, and I'm still sorting out how I think about it. On the one hand, I think Farley makes a really good point. There are ways in which the "standard" narrative leaves some things out. Let a thousand IR blogs bloom!

On the other hand ... well, I'm leery of advising junior faculty and grad students to throw caution into the wind and blog outside the box, as it were. Blogs are becoming more mainstream in international relations scholarship and political science, but I wouldn't describe them as truly mainstream just yet. So I have some residual caution. 

There's something else, however. If blogs are going to occupy a more central role in the field of political science, then they're inevitably going to be measured, assessed, evaluated, and quantified in any kind of professional assessment. That's what happens when people are hired or promoted in the academy. But for blogging, this is problematic, because the distribution of traffic and linkage in blogs is highly asymmetric. I worry that any kind of assessment will skew against the majority of blogs. More generally, I'm kinda dubious about the metrics we do have to measure blogs. This doesn't mean we shouldn't do it -- but I think we need to be aware of the risks going forward, and I think I'm less sanguine about them than Farley.   

Clearly, technology is changing the way we in IR scholarship do business. We're going to need to figure out what that means in the years ahead. 

Developing...

Over the weekend WikiLeaks pulled a pretty silly prank.  Through a combination of some savvy Web design and hacking, the organization managed to convince some people that former New York Times executive editor Bill Keller had written an op-ed column.  In the fake column, fake Keller claimed the Times was under threat of a financial blockade for publishing the Wikileaks cables -- a blockade that Wikileaks itself is experiencing (hence the motivation). 

The prank managed to fool some of the people for a short time, including the New York Times' chief technology reporter.  Just as quickly, however, the fact that the column was a hoax also spread.  For Glenn Greenwald, this is evidence that the Internet is actually a superior fact-checking entity than the traditional mainstream media used to be:

[E]rrors and frauds have a very short life-span on the Internet. The power to tap into collective knowledge and research is so much more potent than being confined to a single journalistic outlet. The ability to have one’s work take the form of a mass dialogue, rather than a stagnant monologue, is incredibly valuable. It is true that the Internet can be used to disseminate falsehoods quickly, but it just as quickly roots them out and exposes them in a way that the traditional model of journalism and its closed, insular, one-way form of communication could never do....

For anyone who still believes that traditional journalism is inherently more reliable than the Internet, just follow the excellent suggestion this morning from Alexa O’Brien: just compare the duration and seriousness of the frauds and fakes enabled by the model of traditional journalism. Long before the Internet — in 1938 — a dramatized radio broadcast by Orson Wells (“The War of the Worlds”) of Martians landing on Earth spawned mass panic. More recently, consider the fraud of Iraqi WMDs and the Saddam-Al Qaeda alliance propagated by the nation’s leading traditional media outlets, or the fraudulent story they perpetrated of how grateful Iraqis spontaneously pulled down the Saddam statue,, or the fraudulent tales they told of Jessica Lynch engaging in a heroic firefight with menacing Iraqis and Pat Tillman standing up to Al Qaeda fighters before they gunned him down. And that’s to say nothing of the Jayson-Blair-type of rogue, outright fabrications.

Those frauds were vastly more harmful than anything the Internet has produced. And they took far longer to expose. That’s because they were disseminated by stagnant, impenetrable media outlets which believe only in talking to themselves and trusting only government sources. Nobody can get away with that on the Internet. The voices are far more diversified, the scrutiny is far more rigorous, the feedback is much more rapid, and the process is much more democratized. Yes, the Internet enabled a fake Bill Keller column to fool some people for a few hours, but — through the work of journalists, experts, and anonymous, uncredentialed users alike — it also immediately exposed the hoax, documented how it happened, and drew rapid lessons from it. The prime lesson is not that Internet journalism is more prone to errors; it’s that it is far more adept and agile at detecting and banishing them.

I've made arguments like this one in the past on the blog, so I'm pretty sympthetic to Greenwald's thesis that the Internet snuffs out "errors and frauds" just as quickly as they are created.  I'd also agree than in a pre-web era, the negative ramifications of mainstream media errors were far greater. 

There is another category to consider, however, which is "myths" -- and here the Internet has juuuust a bit of a problem.  Despite copious amounts of evidence, for example, a disturbingly large number of people believe that Barack Obama was not born in the United States -- and the web is a friendly place for their beliefs.  The same could be said with arguments about whether global warming is a hoax or vaccines cause autism.  Furthermore, simple rebuttals aren't always so simple.  Consider this Conservapedia entry on Barack Obama: 

Obama claims to have been born in Hawaii to Stanley Ann Dunham and Barack Obama Sr. - who had married just six months prior - on August 4, 1961 in Honolulu, Hawaii.^[11] Some contend that this story is a complete fabrication. After many leading conservatives including the leadership of this site and Donald Trump called for Obama to release his birth certificate he did on April 27. Sheriff Joe Arpaio of Maricopa County in Arizona conducted an investigation of the Obama's eligibility and alleged that the "birth certificate" was a fake; however, no charges have been filed. Obama was reportedly assigned a social security number whose area code was assigned to applications coming from zip codes in Connecticut.^[12]

Barack Obama Sr. was not a citizen of the United States. At the time of Obama's birth, Kenya was a British colony, meaning that Obama Sr not only held British citizenship, but passed it on to his son. When Kenya gained independence, Obama and his father both lost British citizenship and gained Kenyan citizenship. Barack Obama was a dual citizen of the United States and Kenya until his Kenyan citizenship automatically expired in 1984, as he had failed to renounce US nationality and swear loyalty to Kenya. Despite having been born with US citizenship through his mother, it has been argued that as he was born with dual nationality, he is not a natural-born citizen of the United States, and thus constitutionally ineligible to become President.

Now, to my knowledge, there is nothing factually incorrect with those two paragraphs.  Rather, it's factually incomplete -- there is little discussion of facts in evidence that prove Obama is a natural-born U.S. citizen.  It's simply written in such a way to allow those individuals predisposed to believe Obama is not an American to draw that conclusion.  This is consistent with Cass Sunstein's argument that the Internet allows people to selectively filter the information they acquire such that their ideologies do not face a rigorous challenge. 

This kind of information problem existed before the internet, as anyone familiar with Ron Paul's newsletters could tell you.  What the Internet permits is an amplification of conspiracy theories that can attract pockets of people that otherwise would never bother to organize.  In a traditional mainstream media environment, it was possible to shoot down any zany conspiracy theory that bubbled up to the surface through more authoritative reporting.  Some people were likely to persist in believing the myth -- but they were less likely to articulate those beliefs to a wide audience. 

I suspect -- and let me stress that this is nothing more than an untested hypothesis -- that speed of transmission is the key variablesthat determines whether the Internet acts as a myth buster or a myth booster.  For "facts" that spread like wildfire, the Internet should work well as a fact-checking engine.  In these moments when there is a great demand for verification, the information ecosystem responds to the "fire alarm" by taking the data point and examining it to within an inch of its life.  The more pervasive the "fact" appears on the web, the greater the supply of people who can likely fact-check it. 

The problem comes with slower-moving facts -- those arguments or statements that are so "out there" that no significant online media would bother to check out until and unless it attracts a large number of devotees.  Myths and conspiracies that spread unchecked for a significant period of time are likely harder to root out.  If myths are given time to grow, then devotees to those myths can also develop defense mechanisms to rebut attempts at fact-checking.  Paradoxically, this kind of myth is more likely to take root if it spreads slowly, requiring a "police patrol" of the Internet to find it.  By the time it is doused with "the truth," there are people who have bought into the myth with sufficient psychological investment that they can tolerate a fair amount of cognitive dissonance. 

So , to repeat myself, I agree with Greenwald that the Internet snuffs out "errors and frauds" just as quickly as they are created.  The problem isn't with the fast-moving memes, however -- it's with the slow-moving ones.  

What do you think? 

For those who believe that political discourse as we know it is in terminal decline, let me quote from a passage I read earlier this week from John Locke's Two Treatieses of Government. It's from his Preface: 

If any one, concerned really for Truth, undertake the Confutation of my Hypothesis, I promise him either to recant my mistake, upon fair Conviction; or to answer his Difficulties. But he must remember two Things;

First, That Cavilling here and there, at some Expression, or little incident of my Discourse, is not an answer to my Book.

Secondly, That I shall not take railing for Arguments, nor think either of these worth my notice: Though I shall always look on my self as bound to give satisfaction to any one who shall appear to be conscientiously scrupulous in the point, and shall shew any just Grounds for his scruples.

For this holiday weekend, I suggest that you read the whole thing. 

Charli Carpenter and I have a co-authored a paper in the August 2010 issue of International Studies Perspectives entitled, "International Relations 2.0: The Implications of New Media for an Old Profession."  It's on the effect that Web 2.0 technologies are having on the study and practice of international relations.  The abstract: 

The International Relations (IR) profession has not fully taken stock of the way in which user-driven information technologies—including Blogger, Facebook, Twitter, YouTube, and Wikipedia—are reshaping our professional activities, our subject matter, and even the constitutive rules of the discipline itself. In this study, we reflect on the ways in which our own roles and identities as IR scholars have evolved since the advent of “Web 2.0”: the second revolution in communications technology that redefined the relationship between producers and consumers of online information. We focus on two types of new media particularly relevant to the practice and the profession of IR: blogs and social networking sites.

If you ask me, it's worth reading just for the Bill Watterson citation. Still, as Charli has already observed:

Of course if scholarly journal lag-time weren't what it is we had written this more recently than thirteen months ago, we'd probably have also talked about the data generation possibilities of tools like Wikileaks.

Carpenter has fortunately plugged this gap with ferocity in the past week -- see here and here. Charli's quick turnaround on this issue should provide a credible signal to readers as to which of us was the brains and which of us was the beauty in this particular joint venture. 

[Her both times?---ed.  Um.... pretty much, yeah.]

Ezra Klein made an interesting observation a few days ago about how opinion journalists read papers by experts:

[T]his is one of the difficulties with analysis. Fairly few political commentators know enough to decide which research papers are methodologically convincing and which aren't. So we often end up touting the papers that sound right, and the papers that sound right are, unsurprisingly, the ones that accord most closely with our view of the world.

One way to weigh competing research papers is to consider the publishing outlet.  Presumably, peer-reviewed articles will carry greater weight.  Except that Megan McArdle doesn't presume:

Especially for papers that rely on empirical work with painstakingly assembled datasets, the only way for peer reviewers to do the kind of thorough vetting that many commentators seem to imagine is implied by the words "peer review" would be to . . . well, go back and re-do the whole thing.  Obviously, this is not what happens.  Peer reviewers check for obvious anomalies, originality, and broad methodological weakness.  They don't replicate the work themselves.  Which means that there is immense space for things to go wrong--intentionally or not....

This is not to say that the peer review system is worthless.  But it's limited.  Peer review doesn't prove that a paper is right; it doesn't even prove that the paper is any good (and it may serve as a gatekeeper that shuts out good, correct papers that don't sit well with the field's current establishment for one reason or another).  All it proves is that the paper has passed the most basic hurdles required to get published--that it be potentially interesting, and not obviously false.  This may commend it to our attention--but not to our instant belief.

This jibes with a recent Chonicle of Higher Education essay that bemoaned the explosion of research articles: 

While brilliant and progressive research continues apace here and there, the amount of redundant, inconsequential, and outright poor research has swelled in recent decades, filling countless pages in journals and monographs. Consider this tally from Science two decades ago: Only 45 percent of the articles published in the 4,500 top scientific journals were cited within the first five years after publication. In recent years, the figure seems to have dropped further. In a 2009 article in Online Information Review, Péter Jacsó found that 40.6 percent of the articles published in the top science and social-science journals (the figures do not include the humanities) were cited in the period 2002 to 2006.

None of this provides much comfort for the layman interested in navigating through the miasma of contradictory research papers.  How can the amateur policy wonk separate the wheat from the chaff? 

Below are seven useful rules of thumb to provide you.  These are not foolproof -- in fact, that's one of the rules -- but they can provide some useful filtering while trying to discern good research from not-so-good research: 

1)  If you can't read the abstract, don't bother with the paper.  Most smart people, including academics, don't like to admit when they don't understand something that they read.  This provides an opening for those who purposefully write obscurant or jargon-filled papers.  If you're befuddled after reading the paper abstract, don't bother with the paper -- a poorly-worded abstract is the first sign of bad writing.  And bad academic writing is commonly linked to bad analytic reasoning. 

2)  It's not the publication, it's the citation count.  If you're trying to determine the relative importance of a paper, enter it into Google Scholar and check out the citation count.  The more a paper is cited, the greater its weight among those in the know.  Now, this doesn't always hold -- sometimes a paper is cited along the lines of, "My findings clearly demonstrate that Drezner's (2007) argument was, like, total horses**t."   Still, for papers that are more than a few years old, the citaion hit count is a useful metric.

3)  Yes, peer review is better.   Nothing Megan McArdle wrote is incorrect.  That said, peer review does provide some useful functions, so the reader doesn't have to.  If nothing else, it's a useful signal that the author thought it could pass muster with critical colleagues.  Now, there are times when a researcher will  bypass peer review to get something published sooner.  That said, in international relations, scholars who publish in non-refereed journals usually have a version of the paper intended for peer review. 

4)  Do you see a strawman?  It's a causally complex world out there.  Any researcher who doesn't test an argument against viable alternatives isn't really interested in whether he's right or not -- he just wants to back up his gut instincts.  A "strawman" is when an author takes the most extreme caricature of the opposing argument as the viable alternative.  If the rival arguments sound absurd when you read about them in the paper, it's probably because the author has no interest in presenting the sane version of them.  Which means you can ignore the paper. 

5)  Are the author's conclusions the only possible conclusions to draw?  Sometimes a paper can rest on solid theory and evidence, but then jump to policy conclusions that seem a bit of a stretch (click here for one example).  If you can reason out different policy conclusions from the theory and data, then don't take the author's conclusions at face value.  To use some jargon, sometimes a paper's positivist conclusions are sound, even if the normative conclusions derived from the positive ones are a bit wobbly.  

6)  Can you falsify the author's argument?    Conduct this exercise when you're done reading a research paper -- can you picture the findings that would force the author to say, "you know what, I can't explain this away -- it turns out my hypothesis was wrong"?  If you can't picture that, then you can discard what you're reading a a piece of agitprop rather than a piece of research. 

7)  Fraudulent papers will still get through the cracks.  Trust is a public good that permeates all scholarship and reportage.  Peer reviewers assume that the author is not making up the data or plagiarizing someone else's idea.  We assume this because if we didn't, peer review would be virtually impossible.  Every once in a while, an unethical author or a reporter will exploit that trust and publish something that's a load of crap.  The good news on this front is that the people who do can't stop themselves from doing it on a regular basis, and eventually they make a mistake.  So the previous rules of thumb don't always work.  The  publishing system is imperfect -- but "imperfect" does not mean the same thing as "fatally flawed." 

With those rules of thumb, go forth and read your research papers. 

Other useful rules of thumb are encouraged in the comments. 

It is not a shock for readers to learn that  I think blogs have been a great innovation for the information ecosystem.  That said, I've come across two examples in the past 24 hours where bloggers have really let down the guild -- because they blogged before reading through to the end.   

Example #1:  Brad DeLong, "Joshua Green Doesn't Get How Government Works.": 

Joshua Green:

Yes, Larry Summers is Leaving: Summers cast his eye on the Fed chairmanship and agreed to bide his time until Ben Bernanke's term ended at the NEC--a staff position well below his old job as Clinton's Treasury secretary...

"Below." "Above." Joshua Green is old enough to know that whether the Secretary of the Treasury is "above" or "below" the Assistant to the President for Economic Policy really depends on the people and on the President....

The rule is that with a President who is weak--or uninterested in economic policy--or when the Chairs of House Banking, Senate Banking, Senate Finance, and House Ways-and-Means are strong, then the Treasury Secretary is the more powerful position. But with a President who is strong and interested in economic policy, the Assistant to the President is the higher-ranking job.

This is an old, old story....

Over the generations, face-time with the king is what matters for power. (emphasis added)

DeLong makes a fair point here -- face time can be really important sometimes.  The thing is, Green tackled the "face-time" question head-on in his original story about Larry Summers: 

As Jonathan Alter lays out in his forthcoming book, "The Promise," Summers maneuvered to sideline people like Paul Volcker, Joe Stiglitz, and even Orszag, behavior more characteristic of the Clinton administration than the Obama administration. Alter also reveals that Obama's nickname for Summers is "Dr. Kevorkian," which does not imply paternal fondness.

But what really makes me believe that Summers won't stick around is that all this Machiavellian intrigue has failed to win him what he wanted most: power. Summers gets plenty of presidential face time, but he's not the nexus of White House activity that everyone expected him to be, and that doesn't sit well according to the Summers associates I spoke with. In my Atlantic piece, I go into considerable detail about how Geithner, and not Summers, came to be the key person on financial matters. But it wasn't just finance. Energy and health care care were also routed elsewhere, to Carol Browner and Nancy Ann DeParle. The hand-holding of anxious lawmakers that became an integral part of the NEC job under Summers's mentor, Bob Rubin, is being handled by another economist, Mark Zandi, a former McCain adviser. Marc points out that Summers does "ride herd over the administration's infrastructure renewal program." But I'd wager that infrastructure renewal is not what Larry Summers pictured for himself when he arrived at the White House.  (emphasis added)

Green clearly thinks that face time is important --- it's just that Summers appears to have a limited ability to converty that face-time into actual policy responsibility.  DeLong can dispute that assertion -- but he can't assert, as he blogged, that Green is somehow naive about the way Washington works.  Based on that paragraph, he seems better clued in than DeLong. 

Example #2:  Conservative bloggers on this AP story about the Obama administration's provisional changes to the National Security Strategy.  The story leads with the following:

President Barack Obama's advisers will remove religious terms such as "Islamic extremism" from the central document outlining the U.S. national security strategy and will use the rewritten document to emphasize that the United States does not view Muslim nations through the lens of terror, counterterrorism officials said.

The change isa significant shift in the National Security Strategy, a document that previously outlined the Bush Doctrine of preventative war and currently states: "The struggle against militant Islamic radicalism is the great ideological conflict of the early years of the 21st century."

As you can imagine, there are a fair number of conservative bloggers lashing out at this story.  Andy McArthy calls it "willful blindness."    Michelle Malkin concludes, "We’ve traded in a reality-based national security strategy for a Birkenstock bumper-sticker fantasy plan." 

Fine, except that if you read the whole AP story, you discover that the origins of the "Birkenstock bumper-sticker fantasy plan" are... Ronald Reagan and George W. Bush: 

Obama's speechwriters have taken inspiration from an unlikely source: former President Ronald Reagan. Visiting communist China in 1984, Reagan spoke to Fudan University in Shanghai about education, space exploration and scientific research.

He discussed freedom and liberty. He never mentioned communism or democracy.

"They didn't look up to the U.S. because we hated communism," said Deputy National Security Adviser Ben Rhodes, Obama's foreign policy speechwriter.

Like Reagan in China, Obama in Cairo made only passing references to terrorism. Instead he focused on cooperation....

Obama did not invent Muslim outreach. President George W. Bush gave the White House its first Quran, hosted its first Iftar dinner to celebrate Ramadan, and loudly stated support for Muslim democracies like Turkey....

[Karen]

Hughes and Juan Zarate, Bush's former deputy national security adviser, said Obama's efforts build on groundwork from Bush's second term, when some of the rhetoric softened. But by then, Zarate said, it was overshadowed by the Guantanamo Bay detention center, the abuses at Abu Ghraib prison and a prolonged Iraq war.

"In some ways, it didn't matter what the president did or said. People weren't going to be listening to him in the way we wanted them to," Zarate said. "The difference is, President Obama had a fresh start."

I'm not saying that this strategy will work -- but Malkin and McArthy don't seem to realize that they're saying that Reagan and Bush were Birkenstrock-wearers as well. 

FP's own Peter Feaver is quoted in the AP story as a skeptic, and I think he makes a fair point.  Over at the Corner, Michael Rubin looked at the same story and suggested flaws in the administration's argument.   I'm only partially convinced by Rubin's argument -- but it's clear that at least Rubin read the whole AP story before blogging about it. 

Seriously, read the whole thing before blogging it. 

I chaired a panel today at the International Studies Association on the role of  blogging in scholarship and policy advocacy.  Participants included FP's Steve Walt, LGM's Rob Farley and Charli Carpenter, IPEatUNC''s  Will Winecoff, and He Who Does Not Need a Blog Joseph Nye.  FP's Peter Feaver, Duck of Minerva's Stephanie Carvin, and many other luminaries were in the audience. 

The result was one of the most enjoyable ISA panels I've experienced -- thanks entirely to the caliber of everyone else in the room. 

For those interested, do check out Alex Parets' detailed play-by-play of the panel.