Thanks to Scott for pointing me to this excellent post by Michael Clarke on the Scholarly Kitchen, about why scientific publishing hasn't been profoundly disrupted by the Internet (unlike many other fields). Clarke identifies the five functions of a scholarly journal as dissemination of an idea, registration of who came up with it first, validation (peer review--much more on that later), filtration as a means of managing information overload, and designation of the value of the work. This final piece is why all researchers want to publish in the top-flight, high impact factor journals that command monopoly prices in the academic health sciences market.
As Clarke shows, the Web does an amazing job with dissemination and registration. Nobody would argue that print does a better job of disseminating an idea than online articles, and it's as easy as noting the time stamp on an article to tell who developed an idea first. If this were all there was to it, open access would have prevailed long ago and there would be no more need for Open Access Week every October. But it's those other functions--validation, registration, and designation--that are more culturally encoded and harder to replace with technology. And so librarians are caught in the middle: between the goal of providing what customers want (we are a service-oriented field), and a recognition that what they often want places us in a financially precarious position.
Clarke cites Don King and Carol Tenopir to argue that the costs of journals aren't that high when compared to the time that journals save researchers who would otherwise have to ferret out all infomation on their own. That is no doubt true. Thus, Clarke argues that improved workflow productivity tools could become even more valuable than journals. Anything that can save real time of overstrapped researchers is precious. So this would mean eventual downward pressure in the prices of online journals, as universities begin to value tools and services that improve productivity. Clarke names semantic and mobile technologies, as well as open data standards, as the seeds of such enhanced productivity services.
So, in summary: there will be incremental rather than disruptive change to the scholarly publishing market, and there will eventually be a "mixed market" between traditional journals and ever-improving productivity tools. Where does this leave librarians?
At the simplest level, this points to a reallocation of the library budget to support this mixed landscape of journals and productivity tools. We'd just be dividing the pie differently, and writing checks to new vendors. This seems deeply unappetizing if it's really all we have to look forward to. If so I'd be hard pressed to argue with anyone who claimed that academic health sciences libraries were nothing more than glorified buyer's clubs. Why have a whole library when all you need is a purchasing office?
---
Fortunately, there's a far more robust and intellectually vigorous future awaiting us if we want to claim it. When discussing peer review, Clarke notes that the current system in which papers are shopped around to peers is relatively recent, a latter 20th century phenomenon (journals have existed since 1665). And my librarian friend Eric Schnell has pointed out that peer review is in good measure an administrative function more than something that really preserves the sanctity and truth of scholarship.
Yet, despite the recent provenance and pedestrian function of "peer review," those two words have become holy totems among researchers and librarians. In our classes we make sure to point students to the checkbox in various databases that limits to peer reviewed journals. We extol peer reviewed research and excoriate Wikipedia. And so on and so forth, all in the belief that "peer review" leads people to wonderfully validated, scientifically rigorous, and profoundly accurate research.
But we're wrong, as physician John Ionnnidis has shown repeatedly over many years. His recent profile in the Atlantic explains why. Despite some high profile examples of fraud, the problem is not really villainous scientists. The problem is that the scientific reward system privileges supposedly "breakthrough" research even though such breakthroughs are very far and few between. Thus: all researchers have an incentive to exaggerate the import of what are usually modest findings, and all of their supposedly impartial "peers" have an incentive to give them a pass because they themselves are trying to succeed within the same flawed system. It's a case of misaligned funding incentives leading to perverse results, all of which is shielded from scrutiny by the all-purpose imprimatur of "peer review."
Given the depth of his critique of the peer review system, Ionnnidis has expected to receive fierce criticism from other researchers. But this hasn't really happened; he tends to receive resigned agreement, according to the Atlantic. He takes this to mean that everyone knows he has a point, but nobody knows what to do about it. Here is our point of leverage, as professional librarians.
If we assume that the peer review system is structurally flawed, what should librarians do about it? Here are some options, arranged from least to most attractive:
- Worst: Continue to glorify the peer review system
- Better but still bad: Stop the glorification but go no further
- Better and now good: Vocally discuss the challenges of peer review system on campus
- Best: Formally challenge funding models/priorities at NIH and elsewhere that sustain peer review flaws. If funding streams shift behavior change will follow
Many people would say that the best option isn't for librarians--for policy types and researchers, maybe, but not for us. We are here to serve, not to challenge. I disagree, respectfully and yet strongly. MLA's Code of Ethics for Health Sciences Librarianship states that one of our roles is to facilitate "informed health care decisions." How can we ethically do this if we know that peer review has systemic flaws?
I used to be much more into open access debates; it seemed to me that "liberating the literature" would solve all other problems. But if we're merely increasing access to flawed research, is that such an accomplishment? Clarke may be right that the functions of scholarly journals won't be disrupted so much as incrementally transformed. Fine. But librarians can be much more assertive about pointing to the flaws in the peer review system and seeking transformative solutions. For my money this would be much rewarding than becoming a 21st century buyer's club.
Recent Comments