The case against (only) using metrics for collection management

I’m currently working with Sarah Sutton at Emporia State University on launching a survey to get a sense of how librarians use altmetrics, including for collection development purposes. (Is it useful to know if a book has been cited in a policy document, even if it’s rarely circulated, so you don’t deaccession it? Can monitoring online activity for all scholarship–even articles that a library doesn’t have subscription access to–help librarians make much quicker purchasing decisions for articles and journals that patrons might request? And so on.)

So it was with a lot of interest that I read Chris Bourg’s “ Infrastructure and Culture: A job talk” yesterday. In the talk (which rightly landed her the position of Director of MIT Libraries, at least in part), Bourg describes how institutional cultures are so important to (among other things) the failure or success of campus scholarly communication initiatives.

Crucially, she talks about the unintended consequences that a culture of quantification can have upon decisions that are made for library collections, especially collections that might not be popular but that might inform important research projects, like this study that uses old, uncirculated volumes to study the evolution of Brazilian Portuguese over time.

I encourage you to read Bourg’s post in its entirety, but wanted to pull out one section in particular that I think is a valuable way to think about assessment and metrics w/r/t library services:

Developing new ways of demonstrating the impact of our services and collections is a way of promoting a culture that values assessment, but also recognizes that the true impact of libraries and librarians is often delayed and too idiosyncratic to show up in most of the standard ROI style assessment tools currently in use.

So while I am a fan of assessment and data-driven decision-making, I think it is critically important that we make sure the data we are using captures the full story of our impact. As a social scientist with experience teaching and consulting on statistics and research methods, I’m committed to making sure that the assessment tools we use in libraries are the right ones, that the data we collect measures what really matters, and that we use methods appropriate to the decisions we want to make.

In this spirit, I ask: what methods are you using to drive collection development at your library? And how might you use metrics (including altmetrics) in a more nuanced way to achieve goals that are in line with your larger library (and institutional) culture?

PS Keep your eyes on your inbox–if you’re a librarian at an R1 institution in the US, I’ll likely be emailing you soon to ask you to participate in our survey on altmetrics and libraries.

Stacy Konkiel
Senior Data Analyst

Stacy Konkiel is a Senior Data Analyst at Altmetric.