Brad's idea of using corb is probably a good one here. But if you wanted to do the spawn approach, it should work. You might have to raise the task queue size (in the task server configuration) and/or put some number (100 or 1000 for example) of items in each spawned task.
-Danny ________________________________________ From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] On Behalf Of Rix, Brad [brad....@flatironssolutions.com] Sent: Saturday, May 14, 2011 12:35 PM To: General MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Batch Update You might look into using Corb (http://developer.marklogic.com/code/corb) This is java based tool that handles threading so that you can control how many to update at one time. You point to an XQuery module for a list of URIs, then another XQuery module that is run on each URI. In your case, your URI list might be generated to be just the list of numbers returned. Then the xquery module for each one would run the insert.xqy with the number as input to each call. While running corb, your task queue will never grow because the Java Corb job manages its own queue. [cid:image001.gif@01CC123B.D8DDB280] Brad Rix Senior Developer +1 (303) 542-2172 | Office +1 (303) 915-2771 | Mobile brad....@flatironssolutions.com http://www.flatironssolutions.com IM: AIM: BradRix MSN: brad...@msn.com From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Abhishek53 S Sent: Saturday, May 14, 2011 1:16 PM To: General@developer.marklogic.com Subject: [MarkLogic Dev General] Batch Update Hi All I need to update millions of records in my marklogic repository. xdmp:spawn is failed to work as at certain point of time the task count in the queue is exceeding 1Lac. The sample query is as follows for $x in (1 to 10000000) return xdmp:spawn("insert.xqy",(xs:QName("filename"),$x)) Thanks in advance! Abhishek Srivastav Systems Engineer Tata Consultancy Services Cell:- +91-9883389968 Mailto: abhishek5...@tcs.com<mailto:abhishek5...@tcs.com> Website: http://www.tcs.com ____________________________________________ Experience certainty. IT Services Business Solutions Outsourcing ____________________________________________ =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you _______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
