Another issue. You (Abhishek) mention Amazon Cloud Dev ... are you running on EC2 ? If so what instance type are you using and what storage options ? EBS ? Ephemeral Storage ? Are you using provisioned IOPS ? How much Memory etc. The range of performance on EC2 can vary by 100x or more depending on how you configure things.
----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected] Phone: +1 812-482-5223 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: [email protected] [mailto:[email protected]] On Behalf Of Damon Feldman Sent: Saturday, December 01, 2012 9:59 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Marklogic Transaction Commit - 1K document Abhishek, First of all - thank you for the good question, with a self-contained query we can use to reproduce it and play with the query. 10,000 documents in one transaction is a lot. I find about 100 documents per commit is a good batch size, and 1,000 can be ok. This may vary based on document size, indexes etc. so I suggest you try different commit sizes. So you would use for $i in 1 to 100 return xdmp:eval(' for $x in (1 to 100) let $xml := element root { element id {$x}, element name {"million doc commit test"} } let $uri := fn:concat("/test/",$x) return xdmp:document-insert($uri,$xml) ') And you should see better performance because each eval runs a separate transaction. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic ________________________________ From: [email protected]<mailto:[email protected]> [[email protected]] On Behalf Of Abhishek53 S [[email protected]] Sent: Saturday, December 01, 2012 12:18 PM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Marklogic Transaction Commit - 1K document Hi All, I have gone through some nice blogs over http://developer.marklogic.com/blog/100k-transactions-qconnewyork-2012 posted by Eric David & Team I have tried to validate the same idea with my ML 6.x Amazon Cloud DEV environment where I have initilly NO document present into database [One forest]. The insert query for 1K document is taking more than 5 mnute. Please let me know if I am wrong somewhere Query---------------------------------------------------------- for $x in (1 to 10000) let $xml := element root { element id {$x}, element name {"million doc commit test"} } let $uri := fn:concat("/test/",$x) return xdmp:document-insert($uri,$xml) Profile Reort 5+ minute --------------------------------------------------------------- Kind Regards Abhishek Srivastav Tata Consultancy Services Cell:- +91-9883389968 Mailto: [email protected]<mailto:[email protected]> Website: http://www.tcs.com<http://www.tcs.com/> ____________________________________________ Experience certainty. IT Services Business Solutions Outsourcing ____________________________________________ =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
