[CODE4LIB] MARC record model to be inserted in mongodb
Could you please give us any suggestions on a data model example regarding a MARC record? The goal is to be able to store it in mongodb, in an efficient way so as to get results with the appropriate queries. thank you in advance
Re: [CODE4LIB] MARC record model to be inserted in mongodb
Hi Dasos, You may be interested to try J-ISIS. It uses Berkeley DB as key/value NoSQL database and Lucene for indexing records. https://kenai.com/projects/j-isis https://kenai.com/projects/j-isis/downloads/download/jisis_suite%2024%20June%202013.zip https://kenai.com/downloads/j-isis/J-ISIS%20Reference%20Manual%2024%20June%202013.pdf J-ISIS uses the MARC record concepts. Data elements are stored in *fields*, each of which is assigned a numeric *tag* indicative of its contents. You may think of the tag as the name of the field as it is known by J-ISIS. The collection of fields containing all data elements of a given unit of information is called a *record*. A field may be optional (i.e. it may be absent in one or more records), it may contain a single data element, or two or more variable length data elements. In the latter case the field is said to contain *subfields*, each of which is identified by a 2-character *subfield delimiter* preceding the corresponding data element. Furthermore a field may be *repeatable*, i.e. any given record may contain more than one instance or *occurrence*, of the field. You can import/export UNIMARC, MARC21, MarcXML, ISO2709, MODS data with different encoding. Once data is loaded in the database, you can index the database, browse, update and search the database. (and much more) J-ISIS is yet under development but is becoming more mature with the last 24 June 2013 release. The Web version is also under development, you can read Web-JISIS document to have an idea. https://kenai.com/projects/j-isis/downloads/download/Web-JISIS.pdf Hope it can help. Best wishes, Jean-Claude Dauphin On Fri, Jul 5, 2013 at 11:47 AM, dasos ili dasos_...@yahoo.gr wrote: Could you please give us any suggestions on a data model example regarding a MARC record? The goal is to be able to store it in mongodb, in an efficient way so as to get results with the appropriate queries. thank you in advance -- Jean-Claude Dauphin jc.daup...@gmail.com jc.daup...@afus.unesco.org http://kenai.com/projects/j-isis/ http://www.unesco.org/isis/ http://www.unesco.org/idams/ http://www.greenstone.org
[CODE4LIB] Σχετ: [CODE4LIB] MARC record model to be inserted in mongodb
thank you all for your quick replies
Re: [CODE4LIB] MARC record model to be inserted in mongodb
Also, I'm sure you have your reasons for wanting (or needing) to use mongodb, but you might want to take a look at this presentation: http://thebuild.com/presentations/pg-as-nosql-pgday-fosdem-2013.pdf and the benchmarks in the latter half of it. -Sean On 7/5/13 5:47 AM, dasos ili dasos_...@yahoo.gr wrote: Could you please give us any suggestions on a data model example regarding a MARC record? The goal is to be able to store it in mongodb, in an efficient way so as to get results with the appropriate queries. thank you in advance
Re: [CODE4LIB] MARC record model to be inserted in mongodb
Have you seen Ross' post:
http://dilettantes.code4lib.org/blog/2010/09/a-proposal-to-serialize-marc-in-json/
?
pymarc can get you this json, e.g.:
```
records = pymrx.parse_xml_to_array('/path/to/some/marc.xml')
json_file = [record.as_json() for record in records]
```
or, for that matter, if you happen to be using Mongo's Python API, you
/may/ be able to call `as_dict()` when you store the record:
```
my_mongo_collection.insert(record.as_dict())
```
It looks like ruby-marc does something similar, and presumably the Mongo
API for Ruby uses Ruby hashes the way that the Python API uses dicts, so
a similar approach is probably possible in Ruby.
As for ...an efficient way so as to get results with the appropriate
queries. I guess that all depends on what you're trying to do.
-Jon
--
Jon Stroop
Digital Initiatives Programmer/Analyst
Princeton University Library
jstr...@princeton.edu
On 07/05/2013 05:47 AM, dasos ili wrote:
Could you please give us any suggestions on a data model example regarding a
MARC record? The goal is to be able to store it in mongodb, in an efficient way
so as to get results with the appropriate queries.
thank you in advance
[CODE4LIB] Σχετ: [CODE4LIB] MARC record model to be inserted in mongodb
i am more familiar to PERL, but your code examples will help a lot,
thank you
Απο: Jon Stroop jstr...@princeton.edu
Προς: CODE4LIB@LISTSERV.ND.EDU
Στάλθηκε: 5:26 μ.μ. Παρασκευή, 5 Ιουλίου 2013
Θέμα: Re: [CODE4LIB] MARC record model to be inserted in mongodb
Have you seen Ross' post:
http://dilettantes.code4lib.org/blog/2010/09/a-proposal-to-serialize-marc-in-json/
?
pymarc can get you this json, e.g.:
```
records = pymrx.parse_xml_to_array('/path/to/some/marc.xml')
json_file = [record.as_json() for record in records]
```
or, for that matter, if you happen to be using Mongo's Python API, you /may/ be
able to call `as_dict()` when you store the record:
```
my_mongo_collection.insert(record.as_dict())
```
It looks like ruby-marc does something similar, and presumably the Mongo API
for Ruby uses Ruby hashes the way that the Python API uses dicts, so a similar
approach is probably possible in Ruby.
As for ...an efficient way so as to get results with the appropriate queries.
I guess that all depends on what you're trying to do.
-Jon
-- Jon Stroop
Digital Initiatives Programmer/Analyst
Princeton University Library
jstr...@princeton.edu
On 07/05/2013 05:47 AM, dasos ili wrote:
Could you please give us any suggestions on a data model example regarding a
MARC record? The goal is to be able to store it in mongodb, in an efficient
way so as to get results with the appropriate queries.
thank you in advance
