Hi all


LibreCat is an open collaboration of the university libraries of Lund, Ghent, 
and Bielefeld to create tools for library and research services. One of
the toolkits we provide is called 'Catmandu' 
(http://search.cpan.org/~nics/Catmandu-0.5004/lib/Catmandu.pm) which is a suite 
of tools to do ETL processing
on library data. We provide tools to import data via JSON, YAML, CSV, MARC, 
SRU, OAI-PMH and more. To transform this data we created a small DSL
language that librarians use in our institutions. Also we make it very easy to 
store the results in MongoDB, ElasticSearch, Solr or export it into various

We create also command line tools because we felt that in our daily jobs we 
were creating the same type of adhoc Perl scripts over and over for endless 

E.g. to create a CSV file of all titles in a MARC export we say something like:

$ catmandu convert MARC to CSV --fix 'marc_map("245","title"); 
retain_field("record");' < records.mrc

To get all titles from our institutional repository we say:

$ catmandu convert OAI --url http://biblio.ugent.be/oai  to JSON --fix 

To store a MARC export into a MongoDB we do:

$ catmandu import MARC to MongoDB --database_name mydb --bag data < records.mrc

Here is a blog post about the commands that are available: 

See our project page for more information about LibreCat and Catmandu : 


and a tutorial how to work with the API


In one of our Catmandu  projects we created a Perl connector for Fedora Commons 
(http://search.cpan.org/~hochsten/Catmandu-FedoraCommons-0.24). One of our 
goals was to integrate
better with the Islandora project. For this we needed a  Perl MODS parser. As 
there was no module available on CPAN we provide a top level module like 
MARC::Record called MODS::Record
http://search.cpan.org/~hochsten/MODS-Record-0.05/lib/MODS/Record.pm. I hope 
this will be of some help for the community. If there are coders here who would 
like to contribute to
the MODS package please drop me a line. I think CPAN MODS support shouldn't be 
dependent on one coder, one institution.

Greetings from a sunny Belgium,

Reply via email to