I am on vacation and traveling this week, and so am taking the opportunity to air again an entry of a couple of years ago on what I called 'stitching costs' ....

We are familiar with switching costs, the costs of changing a supplier. I may decide not to change my phone or email arrangements, for example, because I do not want to incur the effort of notifying all my contacts. Libraries are very familiar with switching costs given the range of data migration issues involved in changing library systems. Indeed, high switching costs are one reason that libraries often stay with the same vendor for long periods.

Libraries are also familiar with high 'integration' costs: perhaps these might be called stitching costs. This means that it may be costly developing higher level services based on integration of various lower level services.

Think for example of the website integration issues libraries have where they want to provide unified access to the catalog, to licensed resources, to repositories and so on. The intermittent levels of integration we see are because the 'stitching costs' are high.

This is largely because they are providing a thin layer over two sets of heterogeneous resources. One is the set of legacy and emerging systems, developed independently rather than as part of an overall library experience, with different fulfillment options, different metadata models, and so on (integrated library system, resolver, knowledge base, repositories, ...). Another is the set of legacy database and repository boundaries that map more to historically evolved publisher configurations and business decisions than to user needs or behaviors (for example, metadata, e-Journals, eBooks, and other types of content, which may be difficult to slice and dice in useful ways).

Or think of higher level federation across library services. We have few compelling federated services, whether these are based on metadata harvesting, metasearch, or other approaches. Again, this is partly because of high stitching costs. I cited Jerry McDonough's article the other day about how abstraction and optionality in library standards design creates unhelpful variation in implementation which in turn is a barrier to efficient interoperability.

Things are changing, as it becomes more important to effectively stitch library resources into other environments and as lighter-weight approaches get adopted. However, it is useful to bear stitching costs in mind when there is discussion of approaches based on federation and interoperability. These costs may be to do with technical issues (interfaces, metadata, ...), or policy issues (ILL policies in a resource sharing system, for example), or higher level organizational and resource allocation issues (who will run the federation service, etc). And they are very real.

[This entry first appeared on October 8, 2008.]

Comments: 3

Jul 18, 2010
Jonathan Rochkind

Sorry to keep plugging it (okay, I'm not really sorry), but my open source Umlaut software is in part intended to deal with 'stitching costs' for one class of services: "known item services".

The idea of Umlaut is that it's a platform that you can easily write plugins for to consult various internal and external sources of data on 'known items' (catalog, link resolver knowledge base, worldcat, Scopus, whatever). The platform is there for you so you can just focus on the source-specific logic in the plugin, is the idea. Then Umlaut vends the aggregated information collected through both an HTML web page, but also several APIs designed to be as easy to use as possible, so you can embed the aggregated information in whatever other services or web pages you want. There is a full api (delivering in XML or json), as well as an API that delivers a collection of HTML snippet sections ready to be dropped in as-is on a foreign web page. Both APIs provide incremental results with information on what services are still fetching data, for polling. (As fetching data from various external services can end up being slow).

So theoretically, once set up, this should decrease both the 'switching' and 'stitching' costs of individual elements in your infrastructure. (Once you've 'stitched' an element in, the 'stitching' cost of it's replacement becomes part of the 'switching' cost too, so these are not unrelated, and people are going to start running into this now that they've 'stitched' a lot of their existing things together). Switch out a source of known item data? No problem, just write an Umlaut plugin for the new one. Switch out a service that consumes known item info from Umlaut and delivers it to users? No problem, the new service just needs to access Umlaut's easy to use APIs.

Now, in practice, it is admittedly not neccesarily _quote_ so simple as I mkae it sound, as writing both writing an Umlaut plugin and making a new product access and deliver info from Umlaut's APIs can be significant tasks -- but I'm convinced that this architecture significantly lowers stitching costs, and switching costs due to stitching costs, in the long run.

As one example, it should theoretically allow us to consider 'link resolvers' just in terms of quality of knowledge base, without worrying about the link resolvers own interface. As Umlaut accesses the link resolver knowledge base via api and provides it's own (html and api) interfaces, if we decide that there's a better product for it's knowledge base, we should be able to write an Umlaut plugin for the better one (as long as it has an api), switch it out, and not only will our user experience be mostly unaffected, all existing 'stitched' services will not have to be touched a bit, they're still talking to Umlaut, which just has a new plugin data source.

Jul 21, 2010
Taylor Dennen

It appears your thinking on federated versus centralized has evolved at least since your 2005 blog entry on this issue, although I cannot see you have taken the issue head on again since then. Where do you see the future (next five years) on this issue, specifically from an end user perspective? Has the sophisitcation of technology (e.g, ability of search engines to index on the fly) evolved sufficiently that federation is the wave of the (immediate) future?

Jul 26, 2010
Lorcan Dempsey

Taylor, I guess I am thinking that federation in the sense of 'metasearch' will give way to a centralised aggregation/index model.

Leave a Comment