catmandu, catmandu::fedoracommons, mods::record,

2013-08-07 Thread Eric Lease Morgan

The three (Perl) modules described below look pretty cool, as well as kewl -- 
an interface to read/write MODS, an interface to interact with Fedora Commons, 
an interface to convert data from one thing to another. I can see how these can 
be useful tools in some of my work. Thank you. --Eric Lease Morgan


On Aug 6, 2013, at 2:59 AM, Patrick Hochstenbach 
patrick.hochstenb...@ugent.be wrote:

 LibreCat
 -=-=-=-=
 
 LibreCat is an open collaboration of the university libraries of Lund,
 Ghent, and Bielefeld to create tools for library and research services.
 One of the toolkits we provide is called 'Catmandu'
 (http://search.cpan.org/~nics/Catmandu-0.5004/lib/Catmandu.pm) which is
 a suite of tools to do ETL processing on library data. We provide tools
 to import data via JSON, YAML, CSV, MARC, SRU, OAI-PMH and more. To
 transform this data we created a small DSL language that librarians use
 in our institutions. Also we make it very easy to store the results in
 MongoDB, ElasticSearch, Solr or export it into various formats.
 
 We create also command line tools because we felt that in our daily jobs
 we were creating the same type of adhoc Perl scripts over and over for
 endless reports. 
 
 E.g. to create a CSV file of all titles in a MARC export we say something 
 like:
 
 $ catmandu convert MARC to CSV --fix 'marc_map(245,title); 
 retain_field(record);'  records.mrc
 
 To get all titles from our institutional repository we say:
 
 $ catmandu convert OAI --url http://biblio.ugent.be/oai  to JSON --fix 
 'retain_field(title)'
 
 To store a MARC export into a MongoDB we do:
 
 $ catmandu import MARC to MongoDB --database_name mydb --bag data  
 records.mrc
 
 Here is a blog post about the commands that are available: 
 http://librecat.org/catmandu/2013/06/21/catmandu-cheat-sheet.html
 
 See our project page for more information about LibreCat and Catmandu : 
 
 http://librecat.org
 
 and a tutorial how to work with the API
 
 http://librecat.org/tutorial/
 
 
 MODS::Record
 -=-=-=-=-=-=
 
 In one of our Catmandu  projects we created a Perl connector for Fedora
 Commons (http://search.cpan.org/~hochsten/Catmandu-FedoraCommons-0.24).
 One of our goals was to integrate better with the Islandora project. For
 this we needed a  Perl MODS parser. As there was no module available on
 CPAN we provide a top level module like MARC::Record called MODS::Record
 http://search.cpan.org/~hochsten/MODS-Record-0.05/lib/MODS/Record.pm. I
 hope this will be of some help for the community. If there are coders
 here who would like to contribute to the MODS package please drop me a
 line. I think CPAN MODS support shouldn't be dependent on one coder, one
 institution.
 
 Greetings from a sunny Belgium,
 Patrick




Re: catmandu, catmandu::fedoracommons, mods::record,

2013-08-07 Thread Paul Poulain
Le 07/08/2013 15:56, Eric Lease Morgan a écrit :
 
 The three (Perl) modules described below look pretty cool, as well as kewl 
 -- an interface to read/write MODS, an interface to interact with Fedora 
 Commons, an interface to convert data from one thing to another. I can see 
 how these can be useful tools in some of my work. Thank you. --Eric Lease 
 Morgan
Yes, they are !

I've cced this mail to my colleague that wrote MARC::Transform, because
both tools could interract.
I also suggested him to see if it would be possible/a good idea to also
be able to store the results into Koha ILS directly. That would be very
helpfull to migrate datas from  to Koha ;-)

(note that he's in vacation for now, so you won't probably get any
feedback before september)

 On Aug 6, 2013, at 2:59 AM, Patrick Hochstenbach 
 patrick.hochstenb...@ugent.be wrote:
 
 LibreCat
 -=-=-=-=

 LibreCat is an open collaboration of the university libraries of Lund,
 Ghent, and Bielefeld to create tools for library and research services.
 One of the toolkits we provide is called 'Catmandu'
 (http://search.cpan.org/~nics/Catmandu-0.5004/lib/Catmandu.pm) which is
 a suite of tools to do ETL processing on library data. We provide tools
 to import data via JSON, YAML, CSV, MARC, SRU, OAI-PMH and more. To
 transform this data we created a small DSL language that librarians use
 in our institutions. Also we make it very easy to store the results in
 MongoDB, ElasticSearch, Solr or export it into various formats.

 We create also command line tools because we felt that in our daily jobs
 we were creating the same type of adhoc Perl scripts over and over for
 endless reports. 

 E.g. to create a CSV file of all titles in a MARC export we say something 
 like:

 $ catmandu convert MARC to CSV --fix 'marc_map(245,title); 
 retain_field(record);'  records.mrc

 To get all titles from our institutional repository we say:

 $ catmandu convert OAI --url http://biblio.ugent.be/oai  to JSON --fix 
 'retain_field(title)'

 To store a MARC export into a MongoDB we do:

 $ catmandu import MARC to MongoDB --database_name mydb --bag data  
 records.mrc

 Here is a blog post about the commands that are available: 
 http://librecat.org/catmandu/2013/06/21/catmandu-cheat-sheet.html

 See our project page for more information about LibreCat and Catmandu : 

 http://librecat.org

 and a tutorial how to work with the API

 http://librecat.org/tutorial/


 MODS::Record
 -=-=-=-=-=-=

 In one of our Catmandu  projects we created a Perl connector for Fedora
 Commons (http://search.cpan.org/~hochsten/Catmandu-FedoraCommons-0.24).
 One of our goals was to integrate better with the Islandora project. For
 this we needed a  Perl MODS parser. As there was no module available on
 CPAN we provide a top level module like MARC::Record called MODS::Record
 http://search.cpan.org/~hochsten/MODS-Record-0.05/lib/MODS/Record.pm. I
 hope this will be of some help for the community. If there are coders
 here who would like to contribute to the MODS package please drop me a
 line. I think CPAN MODS support shouldn't be dependent on one coder, one
 institution.

 Greetings from a sunny Belgium,
 Patrick
 
 


-- 
Paul POULAIN - Associé-gérant
Tel : (33) 4 91 81 35 08
http://www.biblibre.com
Logiciels Libres pour les bibliothèques et les centres de documentation