We are doing a tiny experiment here at Notre Dame with the Internet Archive, 
specifically, we are determining whether or not we can supplement a special 
collection with full text content.

We are hosting at site colloquially called the Catholic Portal -- a collection 
of rare, infrequently held, and uncommon materials of a Catholic nature. [1] 
Much of the content of the Portal is metadata -- MARC and EAD records/files. I 
think the Portal would be more useful if it contained full text content. If it 
did, then indexing would be improved and services against the texts could be 
implemented.

How can we get full text content? This is what we are going to try:

  1. parse out identifying information from
     metadata (author names, titles, dates,
     etc.)

  2. construct a URL in the form of a
     Advanced Search query and send it to the
     Archive

  3. get back a list of matches in an XML
     format

  4. parse the result looking for the "best"
     matches

  5. save Internet Archive keys identifying
     full text items

  6. mirror Internet Archive content locally
     using keys as pointers

  7. update local metadata files pointing to
     Archive content as well as locally
     mirrored content

  8. re-index local metadata

If we are (somewhat) successful, then search results would not only have 
pointers to the physical items, but they would also have pointers to the 
digitized items. Not only could they have pointers to the digitized items, but 
they could also have pointers to "services against the texts" such as make word 
cloud, display concordance, plot word/phrase frequency, etc. These later 
services are spaces where I think there is great potential for librarianship.

Frankly, because of the Portal's collection policy, I don't expect to find very 
much material. On the other hand, the same process could be applied to more 
generic library collections where more content may have already been digitized. 

Wish us luck.

[1] Catholic Portal - http://www.catholicresearch.net/
[2] Advanced search - http://www.archive.org/advancedsearch.php

-- 
Eric Lease Morgan
University of Notre Dame

Reply via email to