Mike, Thank a lot for your inputs.
You were right! The slow query performance (for xdmp update functions) was not related to the merges. We verified by temporarily disabling merging, and found it had none or very little impact on the query performance. Your suggestion on fragmenting the document (we used Fragment Parents) also worked very well. After applying a fragmentations rule (and re-indexing the database) the query performance (for xdmp update functions) improved significantly. We did not notice any negative impact on the data-loading queries. Before Fragmentation (i.e. default Mark logic settings) - Total Size of Database: 25 MB - No of documents: 41 - No. of fragments: 101 After Fragmentation - Total Size of Database: 29 MB (Not sure why this changed) - No of documents: 41 - No. of fragments: 811 Some more questions for you: 1. How does one accurately determine the size of a document (or an element) in Marklogic? I presume that size of an exported XML file on disk is not the same as the size of the same document in Marklogic database? In our application the maximum size of the document on Disk (i.e. exported XML file) is 3.5 MB. 2. Do we still need to consider breaking up our documents (currently 3.5 MB on disk) into smaller pieces? Or does fragment roots/parents have the same effect? Note, in our case the documents are dynamic i.e. the application regularly create/modifies documents in the MarkLogic database. Once again, thanks for all the help. Regards, Deshbir -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Michael Blakeley Sent: Monday, March 16, 2009 10:31 PM To: General Mark Logic Developer Discussion Subject: Re: [MarkLogic Dev General] Peformance issue (Merging) - XDMP:node-replace function Deshbir, You can learn more about merges by reading our admin guide, available via http://developer.marklogic.com/pubs Merges are asynchronous with respect to queries, but they can compete with queries for system resources. I suspect that's a false trail, though. How large is the document on which node-replace is running? If you do see a that ErrorLog extract after every node-replace, that suggests a document size of 5-20 MB. If so, you should consider breaking up your documents into smaller ones, or possibly use a fragment root or fragment parent (fragments are also discussed in the admin guide). -- Mike On 2009-03-16 04:06, [email protected] wrote: > Hello, > > We are experiencing extremely slow XQUERY performance for the > XDMP:node-replace function. Following is an XQUERY snippet that consistently > takes more than 5 secs on one of our servers (Mark Logic 3.2). > ============================================ > let $docbookNode :=<p>hello</p> > let $path := doc(".....")/../../ > return > xdmp:node-replace($path,$docbookNode) > ============================================ > On another (different) Mark Logic installation (3.2), the same code takes > consistently less that 300 milliseconds! > > We've compared the server settings and they appear to be the same across both > servers (they are probably the default Mark Logic installation settings) > > On examining the log folder, we found that every time an "xdmp:node-replace" > was executed, the following lines are being added to the error log file: > ============================================ > 2009-03-16 06:45:32.114 Info: Saving C:\Program > Files\MarkLogic\Data\Forests\Documents\00000470 > 2009-03-16 06:45:32.880 Info: Saved 15 MB in 1 sec at 15 MB/sec to C:\Program > Files\MarkLogic\Data\Forests\Documents\00000470 > 2009-03-16 06:45:33.036 Info: Merging 62 MB from C:\Program > Files\MarkLogic\Data\Forests\Documents\0000046f and C:\Program > Files\MarkLogic\Data\Forests\Documents\00000470 to C:\Program > Files\MarkLogic\Data\Forests\Documents\00000471 > 2009-03-16 06:45:37.661 Info: Merged 55 MB in 5 sec at 11 MB/sec to > C:\Program Files\MarkLogic\Data\Forests\Documents\00000471 > 2009-03-16 06:45:37.958 Info: Deleted C:\Program > Files\MarkLogic\Data\Forests\Documents\0000046f > 2009-03-16 06:45:38.098 Info: Deleted C:\Program > Files\MarkLogic\Data\Forests\Documents\00000470 > ============================================ > > What is going wrong here? What could be causing all the "merging" activity? > > Thanks in advance. > > Regards, > Deshbir > > > > ------------------------------------------------------------------------ > > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
