Kelley,
The work you are proposing is interesting and overlaps somewhat both
with work I have already done and with a new project I'm looking into
here at UVa.
I have been the primary contributor to the Marc4j java project for the
past several years and am the creator of the project SolrMarc which
extracts data from Marc records based on a customizable specification,
to build Solr index records to facilitate rich discovery.
Much of my work on creating and improving these projects has been in
service of my actual job of creating and maintaining the Solr Index
behind our Blacklight-based discovery interface. As a part of that
work I have created custom SolrMarc routines that extract the format of
items similar to what is described in Example 3, including looking in
the leader, 006, 007 and 008 to determine the format "as-coded" but
further looking in the 245 h, 300 and 538 fields to heuristically
determine when the format "as-coded" is incorrect and ought to be
overridden. Most of the heuristic determination is targeted towards
Video material, and was initiated when I found an item that due to a
coding error was listed as a "Video in Braille format".
Further I have developed a set of custom routines that look more closely
at Video items, one of which already extracts the runtime from the
008[18-20] field,
To modify it from its current form that currently returns the runtime in
minutes, to instead return it as HH:MM as specified in your xls file,
and to further handle the edge case of 008[18-20] = "000" to return
"over 16:39" would literally take about 15 minutes.
Another of these custom routines that is more fully-formed, is code for
extracting the Director of a video from the Marc record. It examines
the contents of the fields 245c, 508a, 500a, 505a, 505t, employing
heuristics and targeted natural language processing techniques, to
attempt to correctly extract the "Director". At this point I believe
it achieves better results than a careful cataloger would achieve, even
one who specializes in film and video.
The other project I have just started investigating is an effort to
create and/or flesh out Marc records for video items based on heuristic
matching of title and director and date with data returned from
publicly-accessible movie information sites.
This more recent work may not be relevant to your needs but the custom
extraction routines seem directly applicable to your goals, and may also
provide a template that may make your other goals more easily achievable.
-Robert Haschart
On 12/2/2013 12:37 AM, Kelley McGrath wrote:
I wanted to follow up on my previous post with a couple points.
1. This is probably too late for anybody thinking about applying, but I thought
there may be some general interest. I have put up some more detailed
specifications about what I am hoping to do at
http://pages.uoregon.edu/kelleym/miw/. Data extraction overview.doc is the
general overview and the other files contain supporting documents.
2. I replied some time ago to Heather's offer below about her website that will
connect researchers with volunteer software developers. I have to admit that
looking for volunteer software developers had not really occurred to me.
However, I do have additional things that I would like to do for which I
currently have no funding so if you would be interested in volunteering in the
future, let me know.
Kelley
[email protected]
On Tue, Nov 12, 2013 at 6:33 PM, Heather
Claxton<[email protected]<mailto:[email protected]>> wrote:
Hi Kelley,
I might be able to help in your search. I'm in the process of starting a
website that connects academic researchers with volunteer software
developers. I'm looking for people to post programming projects on the
website once it's launched in late January. I realize that may be a
little late for you, but perhaps the project you mentioned in your PS
("clustering based on title, name, date ect.") would be perfect? The
one caveat is that the website is targeting software developers who wish to
volunteer. Anyway, if you're interested in posting, please send me an
e-mail at [email protected]<mailto:[email protected]>
I would greatly appreciate it.
Oh and of course it would be free to post :) Best of luck in your
hiring process,
Heather Claxton-Douglas
On Mon, Nov 11, 2013 at 9:58 PM, Kelley
McGrath<[email protected]<mailto:[email protected]>> wrote:
I have a small amount of money to work with and am looking for two people
to help with extracting data from MARC records as described below. This is
part of a larger project to develop a FRBR-based data store and discovery
interface for moving images. Our previous work includes a consideration of
the feasibility of the project from a cataloging perspective (
http://www.olacinc.org/drupal/?q=node/27), a prototype end-user interface
(https://blazing-sunset-24.heroku.com/,
https://blazing-sunset-24.heroku.com/page/about) and a web form to
crowdsource the parsing of movie credits (
http://olac-annotator.org/#/about).
Planned work period: six months beginning around the second week of
December (I can be somewhat flexible on the dates if you want to wait and
start after the New Year)
Payment: flat sum of $2500 upon completion of the work
Required skills and knowledge:
* Familiarity with the MARC 21 bibliographic format
* Familiarity with Natural Language Processing concepts (or
willingness to learn)
* Experience with Java, Python, and/or Ruby programming languages
Description of work: Use language and text processing tools and provided
strategies to write code to extract and normalize data in existing MARC
bibliographic records for moving images. Refine code based on feedback from
analysis of results obtained with a sample dataset.
Data to be extracted:
Tasks for Position 1:
Titles (including the main title of the video, uniform titles, variant
titles, series titles, television program titles and titles of contents)
Authors and titles of related works on which an adaptation is based
Duration
Color
Sound vs. silent
Tasks for Position 2:
Format (DVD, VHS, film, online, etc.)
Original language
Country of production
Aspect ratio
Flag for whether a record represents multiple works or not
We have already done some work with dates, names and roles and have a
framework to work in. I have the basic logic for the data extraction
processes, but expect to need some iteration to refine these strategies.
To apply please send me an email at kelleym@uoregon explaining why you
are interested in this project, what relevant experience you would bring
and any other reasons why I should hire you. If you have a preference for
position 1 or 2, let me know (it's not necessary to have a preference). The
deadline for applications is Monday, December 2, 2013. Let me know if you
have any questions.
Thank you for your consideration.
Kelley
PS In the near future, I will also be looking for someone to help with
work clustering based on title, name, date and identifier data from MARC
records. This will not involve any direct interaction with MARC.
Kelley McGrath
Metadata Management Librarian
University of Oregon Libraries
541-346-8232<tel:541-346-8232>
[email protected]<mailto:[email protected]>
