Thoughts on Evidence Based Librarianship: A Screed

Although I once belonged to the editorial board of the journal Evidence Based Library and Information Practice, I am beginning to doubt my allegiance.  These days I find that "evidence based librarianship" is a concept that seems beyond reproach in theory, but--on closer inspection--is problematic for both practical and philosophical reasons.

First things first: how should one define evidence based librarianship?  Let's go to the definition offered by Jon Eldredge, one of the leaders in this movement and a prince of a man. In his influential 2000 paper "Evidence based librarianship: an overview," Jon proposes a seven part conceptual framework for evidence based librarianship:

• EBL seeks to improve library practice by utilizing the best-available evidence combined with a pragmatic perspective developed from working experiences in librarianship;
• EBL applies the best-available evidence, whether based upon either quantitative or qualitative research methods;
• EBL encourages the pursuit of increasingly rigorous research strategies to support decisions affecting library practice;
• EBL values research in all its diverse forms and encourages its communication, preferably through peer-reviewed or other forms of authoritative dissemination;
• EBL represents a global approach to information seeking and knowledge development, involving research but not restricted to research alone;
• EBL supports the adoption of practice guidelines and standards developed by expert committees based upon the best-available evidence, but not as an endorsement of adhering to rigid protocols; and
• In the absence of compelling reasons to pursue another course, EBL adheres to the hierarchy (or levels)...for using the best-available evidence, lending priority to higher levels of evidence from the research.

The EBL movement derives from earlier movements towards evidence based medicine and evidence based health care.  Just as health professionals should use the best available evidence (the more rigorous the better) to make decisions, so too should librarians.

At first blush, it's hard to disagree with these ideas.  Codifying the tenets of evidence based librarianship makes it harder to rely on rules of thumb or what we've always done before.  There is certainly merit in busting through inertia and encouraging all of us to critically examine why we do what we do.

But yet, I worry and wonder that practicing formal evidence based librarianship is often not feasible, and sometimes not even desirable for the question at hand. Jon wrote his article well before the wide adoption of "Web 2.0" tools like blogs, wikis and social networks; as more librarians experiment with these new technologies, the shortcomings of EBL are coming to the fore.

EBL assumes, generally speaking, that external data exists to support or refute a chosen course of action.  But in these heady days of constantly appearing new tools and tricks, we have to make things up as we go along.  At this point in time, looking for evidence of effectiveness for the use of these tools in libraries is a recipe for standing still rather than an aid to better decision making.

Andrew Booth, another EBL thought leader, disagrees with me.  Last year Booth lamented an "evaluation bypass" as libraries throughout the world fired up their blogs, wikis and podcasts. How do we know this is effective? How can we tell if we are reaching our users? Where is the data??  These are all valid questions, but they're all premature at this point in time.  Sometimes we have to learn by doing, rather than learning by examining best practices or lessons learned.  As John Seely Brown notes, "tinkering" is the way to proceed in these Web 2.0 days.

EBL is ultimately a philosophy about how to gather knowledge, more than a position about how to be an effective librarian.  It is empirical, logical, and rooted in the scientific method.  These are all worthy attributes, but they shouldn't be the only arrows in our quiver.  Left brain/right brain; yin/yang; masculine/feminine; emotion/logic--all of these have a place as we grapple to understand the world.  EBL prizes one manner of knowing, and unfairly minimizes the value of others.

Jon might argue that his emphasis on a "pragmatic perspective" in the very first part of the EBL framework should soothe my concerns.  To some extent it does; no theoretical construct can be acted upon with purity, and it's nice to see a recognition of this from the outset.

But how far can we push this? If the practical conditions at any given library regularly trump the imperative to examine processes critically, then EBL becomes meaningless. And the rest of the framework emphasizes reliance on increasingly rigorous evidence. Given that librarianship is still a service oriented more than research based profession, attaining this goal seems very unlikely.

Here's a hypothetical scenario: let's imagine that an academic library decides to cancel its document scanning service for faculty members.  Sure, it's convenient--but the evidence shows that the cost of staffing the service (scanner, personnel, server space) significantly outweighs the revenue generated.  And since an ever greater percentage of material is online, this is a declining part of library operations in any case.  The case is crystal clear.

In some libraries, this evidence would carry the day.  In many others, it would not for all sorts of reasons based on the local context--tradition; influential faculty lobbying; staff resistance; etc.

So EBL is a bit like traditional economic theory, which assumes that people will make rational and logical choices.   As a growing body of research shows, and as the subprime mortgage crisis vividly reveals, this is often not the case.   People are simply not as rational as they probably ought to be; this is the glory and folly of humanity.  And for us librarians, it's why evidence based librarianship will only take us so far.

Comments

[pam]

Hi Marcus,

I find your definitions of EBLIP and librarianship quite narrow.

Librarianship isn’t blogs, wikis, and podcasts. These are just tools. Librarianship involves collection development, information and user services, instruction and pedagogy, technology support, management, human resource management, IT management, project management, building and space planning, outreach, partnership development, advocacy, and a whole range of other activities that depend much from past research and continue to need to be informed by research in the future.

You are correct that EBL grew from EBM but over the past decade or more, the international voices involved in defining a library-specific version of evidence-based practice have modified some of the original definitions you cite to today's Evidence Based Library and Information Practice (EBLIP) definition: “an approach to information practice that promotes the collection, interpretation, and integration of valid, important and applicable user-reported, practitioner-observed and research-derived evidence.” This definition balances both using research, if available, and certainly looking to see what exists as a first point of practice, but also creating it to inform practice and the profession.

We have a responsibility to evaluate, assess, and share our research findings with our profession and EBLIP provides a practical framework and a community of practitioners interested in helping develop that strong research-base in librarianship.

Blogs, wikis, podcasts, and other Web 2.0 technologies have been around for years now, more than enough time for service evaluation and user-based research to have taken place. We’re starting to see some but not near enough. My hope is that the Library 2.0 community is interested in integrating “tinkering” with evidence of meeting user needs and evaluation of new service. Too often first-to-the-post and “innovation” seem to be the goals. Michael Schrage, at ALA’s OCLC Mashed-up Library session last month, defined innovation in an EBL-friendly way, that "Innovation isn't what innovators do....it's what customers and clients adopt." Determining what users have adopted requires research. EBLIP is here to help practitioners, including the Library 2.0 ones, poke at all of the questions librarianship needs answers to.

Cheers,
Pam

[Marcus]

Thanks very much Pam.

My point is not that EBLIP has no role or place, but that it shouldn't be reified as the only way to proceed in our profession. To me, EBLIP is part of the story but not the whole game.

This is ultimately a philosophical debate about what constitutes valid knowledge--librarianship is merely the object under discussion.

With that said, I agree that blogs, wikis, and podcasts are just tools and do not constitute librarianship. But even though they've been around a while I don't think we've reached critical mass of adoption in libraries for evaluation of them yet. This is why I find Andrew Booth's critique to be premature at this point in time.

I used Jon's original framework from 2000 since it's a foundation, but should have been more current. The recent definition of EBLIP presents similar problems, in my view: "an approach to information practice that promotes the collection, interpretation, and integration of valid, important and applicable user-reported, practitioner-observed and research-derived evidence.”

At one level, this makes absolutely perfect sense. But how realistic is this in libraries? Who defines what is important and applicable? And where is the recognition of political realities that can often trump even perfect evidence? This is what I was trying to get at in my document scanning scenario, which had nothing to do with Web 2.0 tools.

Maybe I'm being too hard--EBLIP is just a framework, and everyone knows the real world is messy. But it's a framework with powerful assumptions about what constitutes "truth" behind it, and that's where I object.

I find it hard to think that an everyday librarian will absorb principles of EBLIP into their day-by-day work, just like the notion of a doctor actually utilizing the PICO formula as they gather information is suspect at best. So if EBLIP were presented as a guidepost rather than "the answer," I would have less objection to it. And maybe this is the intent and I'm missing the whole point.

[Denise Koufogiannakis]

Hi Marcus,
I've been involved with EBLIP for about 10 years and am a practicing librarian in an academic environment. It concerns me that you have the impression that EBLIP presents a closed version of truth and the way forward. For me, EBLIP does the opposite. The framework helps to guide my practice and question things on a daily basis. If anything, it shows me that there are no perfect answers, and that "truth" is very complex. Nevertheless, I still want to look to the research as one piece of what guides my decision making process, because it can shed a lot of light on things for me. It opens up my knowledge to go beyond what I already know, or what my colleagues already know. That is why I encourage others to look to the literature, think about what the research tells them, or do their own research when implementing new innovations.

If the EBLIP process has come across as too rigid, perhaps things need to change, and I think that everyone involved in this movement is open to that. I'm certainly not living under false assumptions that practicing librarians go through all these steps or evaluate the literature they read. But I do think that we can all take away some piece of the EBLIP process that might help us do things better or think about things in a different way. I, for one, would never critize someone because they didn't follow the whole process or not use it.

I think that for 99% of us, EBLIP is a guidepost. Maybe the idea has come across to harshly in some cases, for the very reason of making a point or bringing it to people's attention. If that creates improvement via criticism, that's ok; but I hope no one thinks it has to be all or nothing.
Denise

[Marcus]

Thanks so much Denise--what an honor to have two leaders in the EBLIP movement comment on this post!

If EBLIP is really a guidepost rather than "the answer," then I have no objection. But I get a far different sense from the writings. Perhaps this is inevitable byproduct for being an "activist" for a new way of thinking, as you note.

Some of my blog ramblings over the years have been about the "tyranny of the rational," "satisficing" vs. "maximizing," etc. I think EBLIP--at least in perception if not in reality--emphasizes the rational over all other modes of thought. While this might seem rational (ha ha), the truth is that people are far more messy and confused than we want to admit.

So I would encourage EBLIP leaders (you, Pam, Jon, etc.) to counter this perception of "rational primacy," or to argue for why this primacy is justifiable.

Only if you both want to, of course--thanks again for your comments.

[T Scott]

At the EBLIP conference in North Carolina last year, Andrew and I held a "friendly debate" titled: EBLIP -- Clear, Simple and Wrong? It's up on youtube: http://www.youtube.com/watch?v=_iLUMV25oxg

[sarah_mc]

I try to make data-driven decisions in my practice, and I appreciate EBLIP attempts to take librarianship research and publication beyond "this is how we did it at my library".

That said, I found David Berwick's commentary in the March 12 issue of JAMA, "The Science of Improvement," to be a very useful critique of the limitations of standard EBM methods, especially when they are used to study (essentially) social-science phenomena.

Berwick proposes four changes to standard EBM research approaches:

1) embracing a wider range of scientific methodologies;

2) reconsider thresholds for action on evidence (do p values always need to be below .05 to be "significant" when we aren't talking about life-or-death interventions?);

3) rethink views about trust and bias, because interventions proceed within the context of local culture;

4) and finally, academicians must remain respectful of clinical expertise and the drive toward innovation.

Complete citation:
JAMA. 2008 Mar 12;299(10):1182-4. The science of improvement. Berwick DM. PMID: 18334694

It's worth a read.

[pam]

Good points Sarah. Similar discussion has taken place in EBLIP publications too. For just a couple of voices, see:

Given, Lisa (2006). Qualitative research in evidence-based practice: a valuable partnership. Library Hi-Tech 24:3 pp. 376-386

and (shamelessly)

Ryan, Pam (2006). EBL and Library Assessment : Two Solitudes? Evidence Based Library and Information Practice 1(4):pp. 77-80.

Cheers,
Pam