Hi OpenLibrary Tech, Great effort!! Thank you!
At DTU Library, we would very much like to include OL data into our search engine - to enrich the search experience for our users, and to drive (some) traffic for books back to the OL website. We discovered http://openlibrary.org/developers/dumps ; it looks great! For including into our search engine that kind of bibliographical data available online, we have developed robots/agents which are responsible for discovering if new data are available from our various sources (and if so, fetch the new material and ingest it into our processing pipeline, eventually leading to the indexing of that material). Announcing links to compressed dump files on a website (like onhttp://openlibrary.org/developers/dumps) is patrolable by our robots. But rather inconvenient and error prone. Do you have any plans to expose these dump files through other protocols (FTP, http directory listings, etc) more suited for robots/agents? And perhaps accompanied by digests (e.g. MD5) of the files, so robots can easily detect, if there are in fact new/changed material to download)? Any plans to supplement compressed full dumps by using protocols like OAI-PMH (http://www.openarchives.org/OAI/openarchivesprotocol.html) or ResourceSync (http://www.openarchives.org/rs/0.9/toc) to minimize the effort for clients when retrieving incremental updates? Again: Great effort! And thanks in advance! Cheers, Christian Tønsberg, Manager IT systems DTU Library
_______________________________________________ Ol-tech mailing list [email protected] http://mail.archive.org/cgi-bin/mailman/listinfo/ol-tech To unsubscribe from this mailing list, send email to [email protected]
