Hi, Mike.
I don't know of any off-the-shelf software that does de-duplication
of the kind you're describing, but it would be pretty useful. That
would be awesome if someone wanted to build something like that into
marc4j. Has anyone published any good algorithms for de-duping? As I
understand it, if you have two records that are 100% identical except
for holdings information, that's pretty easy. It gets harder when one
record is more complete than the other, and very hard when one record
has even slightly different information than the other, to tell
whether they are the same record and decide whose information to
privilege. Are there any good de-duping guidelines out there? When a
library contracts out the de-duping of their catalog, what kind of
specific guidelines are they expected to provide? Anyone know?
I remember the open library folks were very interested in this
question. Any open library folks on this list? Did that effort to de-
dupe all those contributed marc records ever go anywhere?
Bess
On Oct 20, 2008, at 1:12 PM, Michael Beccaria wrote:
Very cool! I noticed that a feature, MarcDirStreamReader, is
capable of
iterating over all marc record files in a given directory. Does anyone
know of any de-duplicating efforts done with marc4j? For example,
libraries that have similar holdings would have their records merged
into one record with a location tag somewhere. I know places do it
(consortia etc.) but I haven't been able to find a good open program
that handles stuff like that.
Mike Beccaria
Systems Librarian
Head of Digital Initiatives
Paul Smith's College
518.327.6376
[EMAIL PROTECTED]
---
This message may contain confidential information and is intended only
for the individual named. If you are not the named addressee you
should
not disseminate, distribute or copy this e-mail. Please notify the
sender immediately by e-mail if you have received this e-mail by
mistake
and delete this e-mail from your system.
-----Original Message-----
From: Code for Libraries [mailto:[EMAIL PROTECTED] On
Behalf Of
Bess Sadler
Sent: Monday, October 20, 2008 11:12 AM
To: [email protected]
Subject: [CODE4LIB] marc4j 2.4 released
Dear Code4Libbers,
I'm very pleased to announce that for the first time in almost two
years there has been a new release of marc4j. Release 2.4 is a minor
release in the sense that it shouldn't break any existing code, but
it's a major release in the sense that it represents an influx of new
people into the development of this project, and a significant
improvement in marc4j's ability to handle malformed or mis-encoded
marc records.
Release notes are here: http://marc4j.tigris.org/files/documents/
220/44060/changes.txt
And the project website, including download links, is here: http://
marc4j.tigris.org/
We've been using this new marc4j code in solrmarc since solrmarc
started, so if you're using Blacklight or VuFind, you're probably
using it already, just in an unreleased form.
Bravo to Bob Haschart, Wayne Graham, and Bas Peters for making these
improvements to marc4j and getting this release out the door.
Bess
Elizabeth (Bess) Sadler
Research and Development Librarian
Digital Scholarship Services
Box 400129
Alderman Library
University of Virginia
Charlottesville, VA 22904
[EMAIL PROTECTED]
(434) 243-2305
