Big Data is a big trend, but as with expressions for other newly forming areas, it may evoke different things for different people.

A few years ago, academic libraries might have thought of scientific or biomedical data when they heard the expression 'big data'. In particular, the publication of The Fourth Paradigm: data-intensive scientific discovery helped crystallise awareness of developments in scientific practice.

More recently, however, big data has become a much more general term, across various domains. Indeed, it is now common to read about big data in the general business press. One comes across it in government and medicine, and in education. For example, a recent article in Inside Higher Ed talks about 'big data' and 'predictive analytics' in relation to course data and student retention. There are two interesting aspects of this, one, the data, and, two, the management environment ...

The rise of webscale services which handle large amounts of users, transactions and data has made the management of big data a more visible issue. At the same time, as more material is digital, as more business processes are automated, and as more activities shed usage data, organizations are having to cope with greater volume and variety of relatively unstructured data. Analytics, the extraction of intelligence from usage data has become a major activity. Here is a helpful characterization by Edd Dumbill on O'Reilly Radar.

As a catch-all term, "big data" can be pretty nebulous, in the same way that the term "cloud" covers diverse technologies. Input data to big data systems could be chatter from social networks, web server logs, traffic flow sensors, satellite imagery, broadcast audio streams, banking transactions, MP3s of rock music, the content of web pages, scans of government documents, GPS trails, telemetry from automobiles, financial market data, the list goes on. Are these all really the same thing? [What is big data?]

In a brief discussion of big data as a possible trend on FaceBook, Leslie Johnston provided an interesting perspective on issues from the Library of Congress.

Our collections are not just discovered by people and looked at, they are discovered by processes and analyzed using increasingly sophisticated tools in the hands of individual researchers, using just laptops. And we not only have TB/PB of digital collections, we will have billions of items, so fully manual processing/cataloging is rapidly becoming a thing of the past.

Leslie expanded on some of the actual data ...

• 5 million newspaper pages, images with OCR, available via API, used in NSF digging into data project for data mining, combined with other collections used in new visualizations, and in an image analysis project.
• 5 billion files of all types in a single institutional web archive - researchers do not search for and view individual archived sites, they analyze sites over time, and characterize entire corpuses, such as campaign web sites over 10 years.
• Extreme example: over 50 billion tweets: many research requests received to do linguistic analysis, graph analysis, track geographic spread of news stories, etc.
• Collection of 100s of thousands of electronic journal articles, which require article-level discovery: they don't all come with metadata and no one can afford to create it manually.

The remark about manual creation of metadata is one example where current processing methods do not scale. Leslie also notes:

And we cannot do manual individual file QA for mass digitization or catalog web archives or tweets without automated extraction. And when we start talking about video and audio, it all requires automated extraction or processing. I know of one request that we process a video to produce an audio-only track so that a transcript could then be automatically generated. LC has 20 PB of video and audio. Can you imagine what it would take to provide that level of service? Researchers started asking a few years ago to get files so they could do it themselves.

The Library of Congress may be a special case, but other organizations are facing similar issues. We are familiar with discussions about research data curation in university settings. Referring to the university challenge, Leslie then points to another interesting example.

I hear this from research libraries, but also from archives, especially state archives that are mandated to take in all state records, physical and electronic. Email archives are already Big Data for a lot of state archives.

Indeed, national or state institutions with responsibility for public records are reconfiguring organizations and systems to manage large volumes of e-records. My colleague Jackie Dooley pointed me at the recent Presidential Mandate on Managing Government Records which has implications for agencies and NARA.

In this context, it is not surprising that we are seeing a growing interest in data mining across domains (Leslie mentions the 'digging into data' challenge). The term 'data scientist' is cropping up in job ads and position titles. A couple of years ago, Hal Varian's comments on the importance of data and the skills required to analyse it were widely noticed.

The ability to take data - to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it's going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complementary scarce factor is the ability to understand that data and extract value from it. [Hal Varian on how the web challenges managers - reg required]

It is clear from this discussion that existing systems are not well suited to manage and analyse these types of data, and this introduces the second topic, the management environment. Indeed, for Dumbill, this is the defining characteristic of big data:

Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn't fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it.

And alternative ways have been emerging, assisted by the webscale companies who had to face these challenges early on. Google provided MapReduce, described by Edd Dumbill as follows:

The important innovation of MapReduce is the ability to take a query over a dataset, divide it, and run it in parallel over multiple nodes. Distributing the computation solves the issue of data too large to fit onto a single machine. Combine this technique with commodity Linux servers and you have a cost-effective alternative to massive computing arrays. [What is Apache Hadoop]

MapReduce is a central part of Hadoop, whose development was supported by Yahoo, and whose further development is now supported within the Apache Software Foundation.

Hadoop brings the ability to cheaply process large amounts of data, regardless of its structure. By large, we mean from 10-100 gigabytes and above. How is this different from what went before?

Existing enterprise data warehouses and relational databases excel at processing structured data and can store massive amounts of data, though at a cost: This requirement for structure restricts the kinds of data that can be processed, and it imposes an inertia that makes data warehouses unsuited for agile exploration of massive heterogenous data. The amount of effort required to warehouse data often means that valuable data sources in organizations are never mined. This is where Hadoop can make a big difference. [What is Apache Hadoop]

The availability of the Hadoop family of technologies (again, nicely described by Dumbill) and cheap commodity hardware has made processing of large amounts of data more accessible. Cloud options are also emerging, from Amazon, Microsoft and others. Uptake has been rapid.

So, while Hadoop and related technologies have emerged in the context of the Big Data requirements of webscale companies, they are becoming more widely deployed. Their scalability, coupled with lower cost, have made them an attractive option across a range of data processing tasks. They may be used with 'big data' and not so big data.

In this way, my big data trend may more realistically be two trends. We are indeed having to process greater volume and variety of data. The description of data management at the Library of Congress provides some nice examples. Several technologies, notably the Hadoop framework, have emerged as a result of such challenges. However, these are now also finding more broad adoption as they reduce costs and provide greater flexibility.

Coda: In OCLC research we have been using MapReduce for several years and more recently have been using Hadoop. We have been also working with colleagues elsewhere in OCLC as we look at where and how Hadoop might provide benefits.

In parallel with bibliographic data, the library community, notably national libraries, have developed 'authorities' for authors and subjects to facilitate this structure. From our current vantage point, I think we can see three stages in the development of these tools.

1. Label. In the first, subject and name authorities provide lists from which values for relevant fields were chosen. Effectively, they constrain the range and format of subject or name data, providing an agreed text label for a concept or name. Examples are LCSH, Dewey, and the Library of Congress Name Authority File. These provide some structuring devices for local catalogues, but those systems do not exploit the full structure of the authority systems from which the values are taken. Think of what is done, or not done, with classification for example. The classification system may not be used to provide interesting navigation options in the local system, and more than likely is not connected back to the fuller structure of the source scheme. That said, having a consistent label is an advantage, and facilitates matching within and between systems.

2. Data. The second stage is that these authority systems are being considered as resources in themselves, and not just as sources of controlled values for bibliographic description. So, we are seeing the Library of Congress, for example, making LCSH and the Name Authority File available as linked data. OCLC is working with a group of national libraries to synthesize name authority files and make them available as an integrated resource in the VIAF service. FAST has recently been made available in this way. The Digital Author Identifier, a national Dutch system for identifying researchers, is interesting in this context. In this arrangement, there is collaboration between the apparatus for uniquely identifying researchers and the national authority file.

3. Network. In a third stage, as these network-level resources become more richly linkable and as local environments exploit that linking ability it becomes possible to do more. This type of linking has only just begun though, and it will be interesting to see how it develops. In this context, a URI is added to the label, making it actionable and globally unique. As an example, think again of the catalogue. The structuring devices we employ are about structuring relationships *within* the catalogue. This would be turned inside out if we not only imported values, but also linked those labels to those external resources. In this way, the item represented could be re-placed in the broad network of knowledge established by the authority file from which it comes.

Of course, alongside this, they may also link to, or draw data from, other navigational, contextual, identifying or structuring resources such as DBpedia, MusicBrainz or Geonames. These and other reference points are likely to be important webscale identity and knowledge organization services. In a sense, more generally, this has already happened, as people orient themselves by links to Wikipedia, MusicBrainz, IMDB and other network level resources.

As in other areas of our activity, we need to think about how activities whose natural level was once local are now moving up to the network level. And once they are at the network level, they have to live alongside other approaches.

If this were to become more common, there are some implications ...

From records to entities ... we ship data around in 'records', bundles about individual items, and our systems are structured around managing these records. We do not tend to manage data about other things of interest to us to the same extent: authors, places, people, concepts, works, and so on, the types of things we have in authority files. What would happen if we more clearly described an item by linking it to these files? More generally, we can see stronger interest emerging in some of these other entities, personal names especially. Think for example of how Amazon has created people pages or the growing interest in researcher identification. Or of places, as geolocation services take hold. Freebase is creating an 'entity graph' giving IDs to millions of entities (people, places and things).

Much of the library linked data discussion has been about making that local record-based data available in different ways. As interesting is the discussion about what key resources libraries will want to link to, and how they might be sustained. An important question for national libraries and others who manage some of the schemes mentioned above is how to move into this third phase. What would this mean for library systems or for library data of this type? What resources are important? How should they be sustained? To make this concrete, are the name authority files maintained by national libraries fit for purpose in a network world? Does it make sense to limit their scope to authors identified in a particular library workflow, cataloging, and exclude other authors (of articles, for example)? Does it make sense to limit their creation to a restricted group of specialist librarians? And so on ...

Finally, as knowledge organization moves to the network level how do library resource relate to others. Can other services leverage the accumulated investment of the library community, or does it fade. The organized relationship between the Deutsche Nationalbibliothek and Wikipedia in Germany is an interesting example here, where the German Wikipedia explicitly takes advantage of the structuring work done by the DNB. Wikipedia itself is very interesting in this regard, as it has effectively become an 'addressible knowledgebase'. If I want to tell you about a new concept or movement, or refer you to a place, or mention a person, I can send you a Wikipedia link. What would be required for Wikipedia to take advantage of 'knowledge organization' approaches developed in the library community?

Related entry:

• Nostalgia, the Dublin Irish Festival and variant forms of names

And, as a reminder, I am on Twitter at http://www.twitter.com/LorcanD.

I was interested to see Rick Anderson remark on the vocabulary used by my colleague Constance Malpas a while ago. This was in the context of a generous note about Constance's "Cloud-sourcing Research Collections: Managing Print in the Mass-digitized Library Environment." [pdf].

I confess that I giggle and shudder simultaneously at the thought of referring publicly to books in our collection as "inventory that is increasingly devalued as an institutional asset." That kind of business-school-flavored language will, not to put too fine a point on it, utterly freak out significant segments of any university faculty, not to mention library staff. [The Scholarly Kitchen]

The 'business' reference is apt, and I confess that my sense of 'asset' in general conversation has indeed been subtly transformed by the narrower acounting sense. For example in the glossary to Robert C Higgins' Analysis for financial management* we read that an asset is 'Anything with value in exchange'. And turning, as one does, to Wikipedia, I read an accounting definition: 'An asset is a resource controlled by the entity as a result of past events and from which future economic benefits are expected to flow to the entity'.

What is relevant here is the idea that assets are things from which you release value. You expect a return. But assets are not ends in themselves. They are means towards creating value. Of course, this is important because assets have associated costs. Managing collections, for example, is not cost-free.

I remember being struck by some sentences about assets in Higgins' book when I read them first a few years ago:

Some newcomers to finance believe assets are a good thing: the more the better. The reality is just the opposite: Unless a company is about to go out of business, its value is in the income stream it generates, and its assets are simply a necessary means to this end. Indeed, the ideal company would be one that produced income without any assets; then no investment would be required, and returns would be infinite.

Yes, financial metrics lend clarity here, but are not relevent to libraries for whom the question of value is different and less susceptible to measurement.

However, it has been interesting see the growing debate about print 'assets' in libraries. As the pressure to repurpose space grows and as the print collection releases progressively less value in research and learning, there is a growing interest in managing down print assets. Not unexpectedly, this is in parallel with an emerging interest in securing system-wide preservation of the collective print record. (I provide some examples in the related entries listed below.)

It is clear that research libraries no longer see collections as ends in themselves, or they do not necessarily equate the size of the collection with the value of the library. More is not necessarily better. They also recognise the opportunity costs of managing large print collections.

As we rethink collections, I think we are seeing them more as assets in the sense I have discussed here, as investment is driven by a stronger sense of how they will be used to generate value in research and learning. Of course, some libraries have thought this way for longer: think of how a busy public library manages its collection. And of course, some libraries will continue to have a mission-driven responsibility to collect significant portions of the scholarly record, although we will probably see more collective approaches here.

Anyway, to get a sense of what I mean, Rick Anderson's presentations might help ...

Related entries:

• The service turn
• The collections shift
• Managing down collections
• We're not going anywhere ... Ok, we lied ...

* It is always a pleasure to read something that is well written. This is a very nice example of fine technical writing.

I have been interested to see three research libraries look for senior digital library posts in recent months.

• Associate Director for Digital Library Programmes and Information Technologies, Bodleian Libraries, University of Oxford.
• Associate Vice President for Digital Programs and Technology Services, Columbia University Libraries/Information Services.
• Head of Digital Library, Information Services, The University of Edinburgh.

Note: given the nature of these resources, the links may not continue to work indefinitely.

Now, these are different posts in different institutions but there is the common ground that you might expect as research libraries look at creating digital infrastructure, engage with research data needs, explore new modes of scholarly communication, and so on. Each is challenging and interesting and offers a wonderful opportunity to be centrally involved in advancing how libraries support changing research and learning practices.

However, I was struck by something else they have in common. Responsibility for the integrated library system (or library management system) appears to be a part of each post, yet it is not foregrounded in the position description. For these libraries, maybe, the ILS is a necessary part of doing business, but is not the site of major development. Designing and developing digital infrastructure now includes the ILS but is no longer led by it. Or maybe there is some other reason .... ?

Now, considerable time and effort goes into these systems, and they will be reconfigured in coming years. Picking up on my opening remarks though, it is interesting to see where the adverts place the emphasis.

The building, owned by Sony DADC, was also the main HQ for the UK's biggest distributor of independent music, Pias. [More than 1.5m CDs destroyed in Sony warehouse fire]

Interestingly, Sony looked after the stock of more than 150 record labels at the warehouse. According to the BBC story quoted above "As well as CDs, the 20,000 sq m (215,000 sq ft) centre was used to store DVDs, Blu-ray discs and discs used for PlayStation Portable games."

It was depressing reading about the impact on the affected independent labels and artists and the music stores who depend on them (see, for example this NME blog entry and this Guardian article). Several support initiatives have been set up to work with the labels through this difficulty. I immediately thought about the heightened awareness about distribution, supply chain management and risk following the Japanese earthquakes earlier this year (see this NYT story for example).

One of the consequences of the arson is that some of the labels may not re-issue physical formats of the music. It will only be available to consumers online in digital format. See this note on the Buzzin' Fly label, for example:

09.08.11 Buzzin' Fly stock goes up in flames in warehouse fire during London riots London, 13h33 , temperature 21°, humidity 72%, clear. Virtually all Buzzin' Fly and Strange Feeling stock was destroyed in an arson attack on the Sony DADC warehouse in Enfield last night during the London riots. The warehouse contained all records distributed by our distributor, PIAS. Other labels are also badly hit. There are a handful of copies of some releases (and a full download catalogue) left on sale on the Buzzin' Fly online shop, but beyond that it is unlikely much of our stock will ever be repressed if at all. A huge slice of the label's history has been destroyed. [Buzzin' Fly Records]

A couple of things occurred to me. First, it is interesting to see the concentration that has occurred in the physical distribution chain and the vulnerability that caused for the indie labels. Second, I was struck again by how one of the benign consequences of the historic model is that preservation is a function of the physical distribution of materials. As national libraries and others look to maintaining a record of the cultural and scholarly environment in a digital world, the model has changed in ways that we don't yet have a very good perspective on. However, it made me realise that I knew little about how well, or not, libraries and other memory institutions (in this case, this seems an appropriate term) are prepared for the acquisition, management and preservation of digital music.

As some folks will know, Moya Brennan is an Irish singer who was a member of the well-known family group, Clannad. They emerged in the 70s, playing very much in a traditional irish music idiom. As they evolved, they developed a style that was influential in the emergence of the sort of new age, 'celtic' music that became popular and has some well known practitioners.

One of the most vivid memories I have of secondary school is when they put on a concert there (before they were very well known). We had never seen anything like it - songs in Irish, but with a double bass and a harp, long hair, and influences from other types of music.

At that time, Moya was known as Máire Ní Bhraonáin. And indeed, if you look at Worldcat identities page for 'Moya Brennan', you will see 'Brennan, Máire' and 'Ní Bhraonáin, Máire' listed as alternatives. (The latter puts the surname in its Irish form.)

'Moya' is an assumed name, perhaps sounding something like the Irish pronunciation of 'Máire'. 'Moya Brennan' is presumably a more memorable name to have outside Ireland than 'Máire Ní Bhraonáin'.

Worldcat Identities does a reasonable job with related Identities, which you can see and follow in the graphical form at the Worldcat Identities Network page. It pulls together the other members of Clannad - her brothers (Brennans) and uncles (Duggans) - and some others.

Interestingly, however, Identities, drawing on the underlying bib data, does not reveal what many people might feel is her most interesting related identity.

This is her more famous sister, who was briefly part of Clannad, and whose name is Eithne Ni Bhraonain. Eithne is somewhat better known as ....... Enya.

Now, as the relationships in Identities - mined from the bib records - are 'bibliographic', there is no reason that family or other relationships which are not somehow represented in the bibliographic data should be present. In fact, as my colleague Diane Vizine-Goetz pointed out there is a relationship in the bibliographic data, but at a very low threshold. See the names collocated in the statement of responsibility in this Danish record (the names are not consistently controlled in WC).

It seems to me that having the relationships made visible in Identities makes me want to see other important relationships - whether or not they are supported by bibliographic relationships. Not having them there seems to me to be an important limitation. This is because I am seeing things through a names or identity filter and the Identity becomes a subject of interest in itself. We are seeing more of this. Open Library, Amazon, Google Scholar, Trove and others all now present 'author' pages, because authors or creators are important topics of interest in themselves, as well as pivots for searching.

This prompts these brief observations about some inevitable futures if library practices in this area are to continue to be relevant:

• As I suggested the last time I spoke about variant forms of Irish names (Name authorities, crowdsourcing, and Máire Mhac an tSaoi), it would be nice to make suggestions to LC or the National Library of Ireland or the BL or ... However, I have no way of easily doing that. Authorities work - and think NACO here - is a professional activity, hedged around by rules and procedures; it is after all 'authorities' work. However, it would seem sensible to open it up to suggestion and information.
• We now have access to several related online sources of data about people and names. Think of the related Wikipedia, Dbpedia and Freebase for example. It would be good to be able to programmatically mine them to enhance the data libraries manage. In Identities, we match names in Wikipedia and link to them when we are reasonably confident the link is correct, but it is not easy to extract relationship information.
• One could change authorities practice to record significant related people who are not supported by bibliographic relationships. This seems sensible to me, although it runs counter to the trend to reduce effort - unless a parallel mechanism is found to more widely share the task of creation. One probably would not want to record that Lorcan Dempsey is Laurence Dempsey's son; but it would be interesting to know that Martin Amis is Kingsley Amis's son, and not just let a general relationship emerge because they are both the subject of a book, for example. One might note here in passing that it would be nice to consistently type those relationships.
• Finally, the current model in library catalogs is one in which centralised creation of 'knowledge organization' systems (subject/name authorities in particular) is matched by decentralised application of those systems in local environments. There is some consistency, but in general local application don't make best use of the data, there may be some variability in application, and links back to the fuller knowledge systems are not present. As more activities move to the network level, it is worth thinking more seriously about network level knowledge organization also, where those knowledge systems become network level resources which others link to. Think of how VIAF or the authorities the Library of Congress makes available at http://id.loc.gov/ might be used, for example, to provide richer context. Currently, a catalog may include, for example, a name in its controlled form: think of how the catalog could either link to, or actually incorporate, the richer data available in the authority file, or a system like Identities. In the context of this note, this would bring the design and construction of those knowledge organization systems more into focus and encourage some of the approaches mentioned above. This is alongside the work that happens in Wikipedia, or Amazon, or other sources. You can get a sense of this at the U Wisconsin Forward experimental system which pulls data from Freebase, with varying success. See for example this search for Conor Cruise O'Brien. It would be good to see library resources used to enrich results or local data. I will return to the topic of network level knowledge organization in a future post.

(Incidentally, 'knowledge organization' seems a rather antique term .. it would be nice to have a better one.)

The initial motivation for this was to put a graphic display of related Identities into an Identities page. This did not work out and we decided to make it available as a standalone app. The aim is to show how something could be built on top of the Worldcat API and the Worldcat Identities Web Services.

It is a 'mashup'. Which prompted me to think that the peak of the mashup has passed? Or at least we do not hear as much about mashups as before? It is interesting to consider why this is the case, but that is something for a future post perhaps.

Enjoy the Worldcat Identities Network ... Here - pretty randomly - is musician Paul Brady, artist Richard Diebenkorn, organizations The Missouri Botanical Gardens and Los Alamos National Laboratory, academic and writer Seamus Deane, poet Eavan Boland, astronomer Fred Hoyle, the interesting Lincoln Theatre, Columbus, and the boat, Brendan.

It should be noted that there is no editorial assistance here. The data is pulled from Worldcat and Worldcat Identities. Worldcat identities is in turn programmatically mined from Worldcat. It would be nice to show the type of each relationship, but that is inconsistently and intermittently available in the bib data.

If you're ready to turn that goal into an accomplishment, you're ready for stickK. stickK was founded on the principle that creating incentives and assigning accountability are the two most important keys to achieving a goal. Thus, the "Commitment Contract" was born.

These are all examples of the growth of gamification as a design principle, the application of game mechanics to services and systems.

Keas.com, a company aimed at reducing corporate healthcare costs by improving employee wellness, puts the 'power of play' at the centre of its method:

Keas is a social game that that promotes employee wellness. Employees form teams and earn points by making healthy choices each week, like eating better, exercising more, or even getting more sleep. Teams participate in wellness challenges and whoever gets the most points wins!

Keas is especially interesting because its co-founder and CTO is the distinguished Adam Bosworth, who made major contributions to products and to general internet technologies while at Microsoft and BEA, before joining Google to work on Google Health.

Bosworth wrote about gamification and Keas on Techcrunch recently. He quickly identified three major inflection points in the development of computer and network technologies, each bringing about an order of magnitude increase in participation: the emergence of the PC, the emergence of the GUI, and the emergence of the web and mobile phones. The next inflection point, he suggested, was not about numbers but about engagement. Engagement is a central aspect of gamification, and is integral to the examples I gave above. Bosworth points to a gamification site which carries this definition: "Gamification is the integration of Game Mechanics and game-thinking in non-game environments to boost Engagement, Loyalty and Fun!".

He argues that inside the term 'gamification' are three important truths. First, social gaming changes engagement in "deep and dramatic ways" by appealing to the "primitive brain in all of us that wants constant rewards, social recognition and adventure". Second, gamification will accelerate the move from physical to digital. He suggests that Groupon has shown that people like the juxtaposition of shopping and games, and that bricks-and-mortar shopping will have to become more fun if it is to slow the trend to online. And third, he forecasts the complete replacement of PCs by mobile devices within 10 years, noting that "social games lend themselves to this form factor and this form factor is location-aware and constantly with you".

He concludes with an interesting message to developers:

We used to teach information design. Then we taught UI design and UI interaction. But now it will be game mechanics. Within two years (if not already), lack of understanding appointment mechanics, game mechanics and leveling will be as crippling to someone who aspires to design online solutions as it is today for someone who doesn't understand HTML and CSS and AJAX and JQuery.

Most of the examples above draw from health care and wellness, but we can see these techniques in widespread use. The Gamification website, mentioned above, has a variety of examples from different industries.

I was wondering whether such approaches were feasible in a library environment when I came across Lemon Tree, a project at the University of Huddersfield in the UK.

Lemon Tree seeks to increase the use of library resources through a social, game based elearning platform. Users will register with the system and be able to earn points and rewards for interacting with library resources, such as leaving comments and reviews of library books. Integration with other social networks such as Twitter and Facebook will be built into the system.

There is a project blog and it will launch in the Fall (aka Autumn). Something to watch ... it will be interesting to see if incentives are strong enough to encourage strong participation. Andew Walsh, who is managing the project, writes about it briefly in an article about the Library's work with social tools.

The rewards users can gain through Lemon tree are developing as we see what works and what our users enjoy, with a massive range of options possible. We are particularly interested though in engaging those people who we know come into our library, but borrow very few books and rarely access our electronic resources. If we can make it fun for them to use the information resources we have and increase their usage then Lemon Tree will have succeeded for us. [Tweets, texts & trees.]

I see that Andrew uses two of the words - 'engaging' and 'fun' - which were part of the definition of gamification given above. The third - 'Loyalty' - would be good too to see as a result too ...

One of the interesting aspects of the launch was the tie-in with Klout. Klout is one of several services which provide analytics around social media activity. It aims to be the 'standard for influence', tracking social media impact. I have spoken about Klout before (Analysing influence .. the personal reputational hamsterwheel). The business model includes the matching of 'influencers' in a particular area with providers of products and services relevant to that area. The providers may provide the influencers with 'perks' (upgrades, samples, etc) which positively influence their tweeting.

The 'perk' in the case of Spotify was an early free invite, and if the 'influencer' gets 5 additional people to sign up (see here for a link to my invite), then he or she gets a free pass to the premium Spotify service. Apparently, the offer was popular, or popular enough to cause Klout to go down for a while (according to Techcrunch).

Klout CEO Joe Fernandez says that Klout has partnered with Spotify to offer free invites to those Klout users who have hight scores in topics relevant to music/entertainment. We don't know the exact number of invites, but Fernandez says Klout is working with Spotify to scale the invitations further. [Techcrunch]

If my experience of the last few days is typical, the interest in Spotify seems to be driving new members to Klout as much as the other way around. I haven't seen discussion of what has happened to the 'influence' threshold.

While there is debate about what exactly Klout measures, some concern about where this model leads, and while it is still early days, I think that this more 'organic' approach to promotion is interesting. As is a service based on the fungibility of influence.

Klout may or may not be successful, but this is is another example of how social and algorithmic approaches are changing our communications and media landscape. It is also a rather literal example of those 'hidden persuaders'.

(Slightly edited for style 7/25/2011.)

More specifically, it is an appropriate description for those operations, common in a digital library environment, where a set of grants are used to keep a set of people working on a set of projects. Many of us are familiar with such environments, either as participants or observers.

In this context, I was interested to see a similar expression used in a report recently in Ars Technica.

But, "these additional responsibilities--and having to learn the new technologies to execute them--are time-consuming, and come at a cost. In many newsrooms, old-fashioned, shoe-leather reporting--the kind where a reporter goes into the streets and talks to people or probes a government official--has been sometimes replaced by Internet searches."

Thus, those "rolling deadlines" in many newsrooms are increasingly resembling the rapid iteration of the proverbial exercise device invented for the aforementioned cute domestic rodent. The observation was first made by Dean Starkman in a Columbia Journalism Review piece titled "The Hamster Wheel."

The "Hamster Wheel" isn't about speed, the report quotes Starkman as saying. "It's motion for motion's sake... volume without thought. It is news panic, a lack of discipline, an inability to say no." [Has the Internet "hamsterized" journalism?]

Here are are some examples ...

We attended the Columbus 'eco-chic craftacular' a couple of weeks ago, pleasantly located only a couple of minutes walk from our house. I was interested to see 'upcycled' books on at least two stands, where physical books were being repurposed as notebooks and cases. I remember being in the bookstore in the Newberry Library last year, and being interested to see some similar examples. At the time of writing, a search on Etsy.com returns over 3.5K resuls for 'upcycled books' and nearly 7K for 'recycled books (of course, not all of these are in fact recycled books, but a lot are).

The marvellous Caustic Cover Critic has an entry about a new publisher, Notting Hill Editions. He praises the appearance and design of the books and notes ...

Several of the many, MANY, MANY future-of-the-book-in-the-face-of-ebooks pieces that have been cropping up in newspapers and online recently have mentioned that the way for physical books to survive in the future may be for them to be beautifully made objects. Notting Hill Editions are doing this just right--the look and feel of their books remind me of the precepts set out by master printer/typographer Daniel Berkeley Updike in his lovely The Well-Made Book. [Notting Hill Editions]

Peter Hirtle alerted me to the Caustic Cover Critic in a comment on a blog entry of mine which discussed book covers and other artifactual properties of books, in line with the comment in the piece quoted above.

Although, as I noted recently, I hope that we don't see an either/or approach to publishing, where 'beautiful' books feel they don't need an electronic parallel.

[Here are the reissued Virago classics which I discussed in the earlier blog entry. Of course, the original Viragos had those iconic green covers. Note the remarks about the current position of Virago in the entry.]

Finally, and anecdotally, what about artists' books? Again, it seems to me that I see more on this topic, most recently, for example, The Art of the Book in California: Five Contemporary Presses, an exhibition at the Cantor Arts Center at Stanford.

We lived in Bristol for many years, near the University of the West of England's Bower Ashton campus, home of the Department of Art and Design. The Department has a Centre for Fine Print Research, with a specialisation in Artists' Books. I mention it here as it provides an entry point into a range of other materials. See for example, Sarah Bodman's very recent presentation Artists and librarians: making art in and out of books [PDF].

Those responsible for library websites, repositories or other systems are indeed now thinking much more about URL patterns, good titles and descriptions, sitemaps, feeds, and so on. In this context, I think it is useful to think about search engine interoperability as managing resources in ways which promote effective crawling, indexing and ranking by search engines. Thinking, in other words, about the robot users of our systems as well as the human users.

Given the importance of this topic, it is good to see that the Strategic Content Alliance in the UK has made several resources available to provide guidance on these topics.

1. Guide to maximising your online presence (PDF)
2. A checklist for value from the internet (PDF)
3. Reports from the field - Experiences from those 'at the coalface' April 2011 (PDF)

In the high level description of this work they evoke the SEO rationale:

In an age when media, business, government and almost every aspect of modern society vies for the users' attention, how can we ensure that the resources that are being created through public funds reach and engage with their constituent audiences?

The materials summarise good practice as interpreted by the Strategic Content Alliance. They go beyond what is usually thought of as SEO, and stay at a pretty general level. The reports from the 'coal face' are interesting, as site managers talk about their experiences trying to enhance their web presence.

Some of the same ground, with examples of good and bad practice, was covered by Ed Summers of the Library of Congress in a presentation that was well received at a DPLA meeting we both attended recently. Here it is ...

As new measures have become possible in a web environment they too have been taken up. See for example the 'top of' lists at SSRN or the elaborate set of rankings from RePEc.

And we are also very familiar with the interest in such measures from individual researchers (it would be interesting to see some analysis of disciplinary differences here - are physicists more interested in their H-Index than geographers?). It is interesting to see influential economics blogger Gregory N Mankiw reference the RePEc list from time to time, for example, as well as other rankings.

As we began to use the web for personal interaction, new measures arose. I am thinking in particular of Technorati and blog rankings which were of interest for a while, although less so now as the blogging environment has changed.

And now we have Twitter. I have been looking at Klout.com recently ... with a mixture of intrigue and dread. Klout describes itself as the "standard for influence" and assigns scores using a proprietary algorithm based on Twitter and Facebook interactions and relationships. What is especially interesting about it is how it is trying to leverage the results of its analysis. An interview with founder Joe Fernandez provides some context. "If Klout CEO and co-founder Joe Fernandez has his way, the Klout score will become a new way of measuring people and their influence online." He makes a comparison with credit ratings:

"Klout is basically your social credit score," Fernandez says. "Consumers should care because it affects the way employers, companies and everyone looks at your ability to spread information as a critical part of the attention economy today."

They have an interesting business model.

Then Klout matches relevant influencers to relevant brands, which provide special offers, especially perks like previews on new product launches. For example if you are influential about technology, HP might send you a free laptop. Or you could get a free upgrade at your favorite hotel.

The idea is that these influencers will create social media buzz that can help a product take off. (People have to sign up with Klout to receive the offers.)

Klout seems to be coming over my horizon more often. I have seen several stories about its use in recruitment, where a person's Klout score is a factor in assessment. Given this interest, it is only natural that there are also critiques of this type of twitter analysis approach in general and of Klout in particular. A moment with Google for example reveals this and this.

Other services exist: Twitalyzer ("serious analytics for social business") for example and the interesting PeerIndex ("understand your social capital").

It is clear that personal or social analytics - based on Twitter and other services - is of great interest, and several companies are exploring how to monetise the measurement of personal reputation or influence. The Klout model suggests how such influence might be fungible and gestures to a future in which one's score might lead to preferential access to services or products. This is an unsurprising development in a world where data about more and more of what we do is mined to provide intelligence for other services (what we spend for example, or where we go on the web).

I need to go now and catch up on my tweeting. I can't let my score drop ... I wonder could it lead to an upgrade on my next transatlantic flight :-)

This reminded me of an entry I wrote a while ago about sourcing and scaling (which is referenced in the article). In a shared network environment, one of the most interesting issues facing libraries is how appropriately to source and scale activities.

A few years ago, this activity would have been sourced within the institution: each library would have developed its own infrastructure, user interface, local community outreach, and so on. Now, such an impulse is questioned. It makes sense to source something like this collaboratively. And it is provided at the level of the network: its target user population is the population of web users.

Questions about sourcing and scaling are becoming much more common as the logic of the network reconfigures patterns of information production and use. What does it make sense to do at institutional level? What does it make sense to source elsewhere (repository services in the cloud, for example, or institutional email services from Google)? And what should be left entirely to other providers? At what level, or scale, is it best to do things? Locally, or within a consortium, or ....?

Think of four sourcing options: Self (provide it locally), collaborative (provide it within a group), public (provided through state or national activity), or third party (provided by another commerical or non-commercial entity).

Think of three scaling options: local or institutional, group, and web scale.

These can be put together to give a variety of options. So, for example, Tripod, the shared catalog of Swarthmore, Haverford College and Bryn Mawr, is a collaboratively sourced group solution. PubMed is a web scale public offering. And, as already noted, Hathi Trust is a collaboratively sourced web scale service.

An interesting contrast between the US and many other parts of the world is that often what is done collaboratively in the US may be done through a public agency elsewhere. For example, Christenson contrasts the HathiTrust as a collaborative activity with something that the JISC, an activity of the public higher education funding councils, might provide in the UK. It is also common in many countries outside the US to have publicly supported union catalogue and related activities.

We can observe two trends. First, there is a trend towards externalisation: libraries are looking to collaboratively source activities or to outsource them to third parties. Think of collaborative activities around managing down print collections here, the West project for example, or the growth of shared library systems (the Orbis Cascade Alliance, for example, recently issued an RFI about a shared integrated library system). Think of the growing interest in cloud-based sourcing of systems and services.

Second, there is a trend to 'move up' in the network, by doing more things at group level within consortia or public contexts (think of OhioLink or Summit), or by leveraging network level services (think of social networking sites, for example).

The current economic environment further encourages these trends. Institutions look for economies of scale through collaboration. And they also want to focus attention on high value areas, and outsource routine or shared activities.

I was reminded of these issues while reading a very interesting internal report of the University of California on library services. This is the interim report of the systemwide Library Planning Task Force, convened under the auspices of the Systemwide Library and Scholarly Information Advisory Committee (splashpage, PDF of report and related material).

A stark environmental picture is presented:

• The libraries will experience budget reductions of as much as $52M or 21% of their current budget base over the next six years. "To put this into perspective, this cut is greater than the total library budget of any single UC campus, and roughly equivalent to the budgets of three of our mid-sized campuses, all AAU members."
• The libraries will likely lose the equivalent of $17M in buying power in the same period given publisher price increases. "This is equivalent to the current library materials budgets of two mid-sized campuses, and means a reduction in the systemwide acquisition rate of about 200,000 items per year."
• Existing facilities will run out of space for new materials over the next 5-7 years, at the same time as "demand increases for extended hours and services and technologically well-equipped and flexible learning environments in the libraries' prime campus locations".

They go on to observe that the impact of these factors can be mitigated through collaboration. They propose four strategies:

1. Expand and collectively manage shared library services.
2. Support faculty efforts to change the system of scholarly communication.
3. Explore new sources of revenue.
4. Improve the existing framework for systemwide planning, consultation, and decision-making.

Of course, the University of California is an unusual institution, bringing together some of the world's major universities in a shared organizational framework. One result of this shared framework has been the California Digital Library, which concentrates operational and innovation capacity for the whole system. CDL has been responsible for some major services, and is an active partner in the Hathi Trust. Another is the Regional Library Facilities, north and south, for managing print collections. A major recommendation is that the range of such shared services should grow, whether sourced within the UC universities or externally.

Cooperation is difficult. Especially where money flows, and impact needs to be seen, at the institutional level. However, given the existing level of shared services, the organizational framework, and the pressures described in this report, it will be interesting to watch what services the UC libraries move to a shared environment over the next few years.

p.s. The report describes the Worldcat Local-based Next Generation Melvyl described in these terms:

The Next‐Generation Melvyl (NGM) initiative moves the discovery of information for researchers and students to the highest networked level. The initiative takes access to the highest level of aggregation and is vital for the most effective provision of information access and services. Strategically, NGM also positions the UC Libraries to provide aggregated access to a significantly increasing array of full‐text information resources: e.g., the millions of digitized books in the Google Books Project and the HathiTrust.