Thanks Danny,
I have used xdmp:eval to do this. I am able to spawn new transaction, I am able
to see files generated as the query executes. Thanks anyways. I haven't read
about xdmp:spawn, please let me know if I miss anything.
for $index in (1 to 3)
let $newDocId:=concat("/item/item", $index,".xml")
return xdmp:eval('
define variable $index as xs:string external
define variable $newDocId as xs:string external
concat($index, $newDocId)',((xs:QName("index"), string($index)),
(xs:QName("newDocId"), string($newDocId)))
, <options xmlns="xdmp:eval">
<isolation>different-transaction</isolation>
<prevent-deadlocks>true</prevent-deadlocks>
</options>)
Regards,
Sundeep
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Danny Sokolsky
Sent: Thursday, January 29, 2009 12:12 AM
To: General Mark Logic Developer Discussion
Subject: RE: [MarkLogic Dev General] XDMP-MEMORY: Memory exhausted
Hi Sundeep,
Your query is loading a million documents in a single transaction, so it
is not surprising that you are running out of memory. I would recommend
breaking this up into smaller transactions. One way to do this is to
use xdmp:spawn to put these on the task server to load asynchronously.
To do that, you can create an xqy file to call and have your for loop
generate a bunch of xdmp:spawn calls, sending the parameters as external
variables.
Another way is to use a program like record loader.
-Danny
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
Sundeep_Raikhelkar
Sent: Tuesday, January 27, 2009 3:55 AM
To: General Mark Logic Developer Discussion
Subject: [MarkLogic Dev General] XDMP-MEMORY: Memory exhausted
Hi,
I am trying to bulk load data into marklogic server using XQuery I
provided below. I am getting error XDMP-MEMORY: Memory exhausted in
/cq/eval.xqy, on line xxx. I have two forests attached to my database
and I have 15 GB free space on the server for these forests to grow. Can
you please elaborate on this issue? What is causing this memory bottle
neck?
for $x in (1 to 1000000)
let $newDocId:=concat("/item/item", $x,".xml")
return xdmp:document-insert($newDocId,
<asset type="item">
<assetType>Item</assetType>
<assetId>{$x}</assetId>
<asset-ref>
<ref-id>{math:fmod($x, 10000)}</ref-id>
</asset-ref>
</asset>)
Regards,
Sundeep
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
solely
for the use of the addressee(s). If you are not the intended recipient,
please
notify the sender by e-mail and delete the original message. Further,
you are not
to copy, disclose, or distribute this e-mail or its contents to any
other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys
has taken
every reasonable precaution to minimize this risk, but is not liable for
any damage
you may sustain as a result of any virus in this e-mail. You should
carry out your
own virus checks before opening the e-mail or attachment. Infosys
reserves the
right to monitor and review the content of all messages sent to or from
this e-mail
address. Messages sent to or from this e-mail address may be stored on
the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
