Hi
We are trying to load 14 million xml files in Mark logic database. The below
xdmp:document-load script could load maximum 5000 xml files at a time.
Anything more than 5000 xml files threw Memory exceptions.
xquery version "1.0-ml";
let $files:=xdmp:filesystem-directory("/filePath/")
for $filepath in $files//dir:entry[1 to 5000]
return (xdmp:document-load($filepath//dir:pathname,
<options xmlns="xdmp:document-load">
<uri>{$filepath//dir:filename/text()}</uri>
<permissions>{xdmp:default-permissions()}</permissions>
<format>xml</format>
<repair>none</repair>
</options>))
Is there any configuration changes required in admin setting to load all the
14 million xml files in 3 to 4 hours?. The total size of the content will be
around 4GB and we have Unix server with 250 GB memory (RAM)
It would be great, if you suggest an best approach to load all the 14
million xml files in the time frame of 3-4 hours.
Thanks and Regards
Rajesh
_______________________________________________
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
