> From: Code for Libraries [mailto:[email protected]] On Behalf Of
> Benjamin Young
> Sent: Monday, March 08, 2010 09:32 AM
> To: [email protected]
> Subject: Re: [CODE4LIB] Q: XML2JSON converter
>
> Rather than using a newline-delimited format (the whole of which would
> not together be considered a valid JSON object) why not use the JSON
> array format with or without new lines? Something like:
>
> [{"key":"value"}, {"key","value"}]
>
> You could include new line delimiters after the "," if you needed to
> make pre-parsing easier (in a streaming context), but may be able to
> get
> away with just looking for the next "," or "]" after each valid JSON
> object.
>
> That would allow the entire stream, if desired, to be saved to disk and
> read in as a single JSON object, or the same API to serve smaller JSON
> collections in a JSON standard way.
I think we just went around full circle again. There appear to be two distinct
use cases when dealing with MARC collections. The first conforms to the ECMA
262 JSON subset. Which is what you described, above:
[ { "key" : "value" }, { "key" : "value" } ]
its media type should be specified as application/json.
The second use case, which there was some discussion between Bill Dueber and
myself, is a newline delimited format where the JSON array specifiers are
omitted and the objects are specified one per line without commas separating
objects. The misunderstanding between Bill and I was that this "malformed"
JSON was being sent as media type application/json which is not what he was
proposing and I misunderstood. This newline delimited JSON appears to be an
import/export format in both CouchDB and MongoDB.
In the FAST work I'm doing I'm probably going to take an alternate approach to
generating our 10,000 MARC record collection files for download. The approach
I'm going to take is to create valid JSON but make it easier for the CouchDB
and MongoDB folks to import the collection of records. The format will be:
[
{ "key" : "value" }
,
{ "key" : "value" }
]
the objects will be one per line, but the array specifier and comma delimiters
between objects will appear on a separate line. This would allow the CouchDB
and MongoDB folks to run a simple sed script on the file before import:
sed -e '/^.$/D' file.json > file.txt
or if they are reading the data as a raw text file, they can just ignore all
lines that start with opening brace, comma, or closing brace, or alternately
only process lines starting with an opening brace.
However, this doesn't mean that I'm balking on pursuing a separate media type
specific to the library community that specifies a specific MARC JSON
serialization encoded as a single line.
I see multiple steps here with the first being a consensus on serializing MARC
(ISO 2709) in JSON. Which begins with me documenting it so people can throw
some darts at. I don't think what we are proposing is controversial, but it's
beneficial to have a variety of perspectives as input.
Andy.
