Hi, I've found the documentation that explains how to use a WritableStream to get /one/ document into MarkLogic, but I couldn't find any example where it shows how one could stream /many thousands/ of documents.
The idea is to load a CSV file with > 1M lines as a ReadableStream and csv-parse and on each "readable" event to push the corresponding JSON object as a document into MarkLogic. The signature for the db.documents.createWriteStream [1] seems to require a document URI to be present at the time of the stream creation, which I cannot supply at the stage of stream creation. The example given in the documentation on how to load many documents doesn't really scale to "big data proportions" ... [2]. Thanks for any help. cheers, Jakob. [1] https://github.com/marklogic/node-client-api/blob/master/lib/documents.js#L468 [2] http://docs.marklogic.com/guide/node-dev/documents#id_18341
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
