Peer review is only a first, preliminary hurdle for a paper to cross; passing peer review and getting published does not mean that your work is right. Some incredibly awful papers get through the review process, somehow. Getting published only means that now your paper is going to be opened up to wider criticism. Don't take the attitude that publication means vindication; I know reviewers, and I've reviewed papers, and I know that reviewers are sometimes lazy, sometimes susceptible to croneyism, and always overworked, and that publication doesn't mean you are right. (my emphasis).Boy, don't I know it! Publishing paleoanthropologically-themed research can sometimes feel like you're entering the Thunderdome, a lesson I learned first-hand in my very first peer-reviewed publication! And Myers makes a good point: when something gets published, it only means that it's serious enough that other experts in the field recognize it's worth discussing, that other professionals can now have a go at it, if you will. Actually you should read the whole post, which is about how people think of blogs, comments on blogs, and the contribution this can make to scientific discourse. It's a thoughtful piece, and one in which PZ - whose writing style can occasionally be, errr, fiery - cogently lays out the case for the usefulness of blogging to make scientific research and its conclusions accessible to a wider audience.
And as for the writing style of blog posts, I also especially liked this quote (which came in second place for quote of the day): "Explain your answers as you would to an undergrad or bright high school student. If you can't, it implies that you aren't looking for an equal opportunity, you are looking for a way to avoid probing questions." Now, that doesn't mean you need to 'dumb it down' or that everybody needs to do this. In my view, it highlights how the web can be used to democratize and favor the diffusion of science. After all, in a time when funding for scientific research increasingly comes from public agencies, it is important to present our work in a manner that is accessible to a well-educated public. This is part of why I have this humble little blog here.
And because common ideas seem to strike different people at the same time, check out John Hawks' recent post on the issue. His take? "I wanted to point to Orac's post, since it is what the blogosphere is the best for: clear, factual critiques that focus on methods." And it's fitting that I mention Hawks here, since he was one of the parties in one of the best 'real-time' virtual debates on Early Upper Paleolithic chronology, a topic near and dear to my heart. On January 12, 2007, John blogged about a new Science paper on the EUP chronology of Kostenki, Russia, criticizing some aspects of the paper's conclusions. Four days later, on January 16, 2007, one of that study's authors, John Hoffecker wrote a reply to Hawks' post, posting it on Kris Hirst's Archaeology Blog, addressing in a detailed manner some of John's critiques. And two days later, on January 18, you guessed it, Hawks had also posted a lengthy reply to some of the points raised in Hoffecker's guest post, situating the debate in a broader context to highlight the significance of the paper he originally blogged about.
OK, lengthy summary to basically highlight that, in the span of just over one week (literally, a blink of an eye in 'journal time'), you had a serious, thought-provoking and scholarly back-and-forth debate on a paper that had just been published. If this had taken place in Science itself, it would have probably taken at least 6 months to get published, far after the paper had faded from public consciousness. Now, this would be all and well for the paleoanthropological community, but for the public at large who might have been interested in the issue, it probably provided a rarely seen instance of how scientists can disagree on details (the specifics of the chronology of the 'transition') while nonetheless agreeing on fundamental issues (that the transition was an important process in recent human evolution). So, yes, blogging can and does serve a very useful purpose in the discourse of science.
4 comments:
Blogs are nice that way. You get comments relatively quickly. I've learned that on my own blog. Of course, I know nothing about scholarly back-and-forth blogging, except what I've seen on the Hawks blog, which I visit almost every day, but the principle, I think, is much the same. Thanks for reminding me.
Hi Julian,
I agree, blogging is certainly an efficient way to allow academics to discuss certain topics. However, the problem arises when academics use blogging as a way to sway colleagues or the public by misinterpreting or misrepresenting peer-reviewed research either intentionally or through laziness. I won't give examples, but it happens.
So, Myers is correct: "publication doesn't mean you are right." However, for a critique/debate to be valid scientifically, it must go through the same tough peer-review process as the original research paper and THEN "be opened up to wider criticism."
In my opinion, I think blogging for clarification and understanding of a paper is great. Blogging for critique is not. My research on rumor psychology (which I will one day get around to publishing, Scanlon and Eren 2008) has shown me that critique belongs in a peer-reviewed context - otherwise it ceases to be critique, and simply becomes a rumor-generating complaint unproductive for the purposes of scientific progress.
Anne - the speed at which exchanges can proceed is definitely one of the most interesting aspect of scientific blogging.
Metin - I hadn't really thought about it this way, though it's certainly true that unfortunate instances of 'blogging for critique' can and do happen, especially, I think, when a paper gets a lot of press in the mainstream media, which tend to gloss over more technical, but crucial aspects of the research. I feel that sometimes bloggers will respond more to how a given paper is discussed in the media than on the paper itself, which is fine, until the paper and its coverage get conflated, which certainly happens.
That said, I think that one of the interesting aspects of blogging is that authors whose work is being commented on also have the opportunity to reply to blog comments on their work in the same arena as the comments were usually put forth. This is why I have comments open for all of my posts - if I feel strongly enough to write something about someone else's work, then I should feel strongly enough to be able to defend those comments.
In that context, I think that one of the main issues is that of tone/flippancy: for better or worse, people tend to be less diplomatic when writing online, which can lead to the posting of potential useful commentary in a form that can easily devolve into ad hominem attacks. It's a balancing act, really, but one that I would argue is not necessarily that different from what you can get from hecklers at professional meetings. Some of the best advice I ever got in my academic training was from an adviser who would mark whole sections of papers, manuscripts and dissertations with the label "save for discussion over beers." Just because some aspects of a paper can be construed as problematic doesn't mean they necessarily should be brought up in scientific discussion - the key is to be able to hone in on why a given study went through peer-review and build on that. At least, that's the logic that I try to apply on AVRPI, albeit perhaps not always as successfully as I strive to.
Certainly, one should be ready to go 'on the record' (i.e., in the literature) if they feel that serious issues are not addressed in a given published study. Then, it gets incorporated into the 'official' recorded history of a debate. I've often wondered about what will become of current internet debates in a generation or two... will people somehow refer to them, or will they just fade into the background? This is why, when possible, I prefer to have commentary sent to journals than just posted online. In this sense, blogging about a given topic is really just a manner of organizing my first thoughts about it, and a chance to incorporate some outside feedback into it.
Evenhanded, constructive, and positive responses (like the ones above) are reasons why I and everyone else continue to visit AVRPI! I think you are right - the issue does become about tone/flippancy, but fortunately there are blogs like this one that are exemplary.
Thanks Julien.
Post a Comment