HI. The first part of your statement says that you loop through your docs.. Do you actually iterate through these or do you get your resultset by way of a search? If you are iterating, then perhaps this is the first bottleneck.
Second, i'm not sure that you really need to bring things back into java? (Maybe I am overlooking somthing, but I'm not a user of the java API anyway). If I understand what you are trying to do, my approach would be in xQuery to pass the results of the first isolating query(optimized vi indexes and not iterating over all docs) into xdmp:spawn[referencing your update code] to manage the updates of these in batches. I will be away form mail for the next day. Hopefully I have not confused things more than helped.. Regards, David On 4 March 2014 08:12, David Ennis <[email protected]> wrote: > HI. > > The first part of your statement says that you loop through your docs.. > Do you actually iterate through these or do you get your resultset by way > of a search? If you are iterating, then perhaps this is the first > bottleneck. > > Second, i'm not sure that you really need to bring things back into java? > (Maybe I am overlooking somthing, but I'm not a user of the java API > anyway). If I understand what you are trying to do, my approach would be > in xQuery to pass the results of the first isolating query(optimized vi > indexes and not iterating over all docs) into xdmp:spawn[referencing your > update code] to manage the updates of these in batches. > > I will be away form mail for the next day. Hopefully I have not confused > things more than helped.. > > Regards, > David > > > On 04/03/14 07:31, ShaikUmmerFaruk D wrote: > > Hi, > > > > We are facing performance issue while fetching documents from Marklogic, > Please let us know your suggestions. We are using Java and Marklogic here. > PFB the detailed scenario. > > > > - We have large volume of xmls in a directory say > /TEST/EMPLOYEE. > > - Here, any two xmls are said to be related if they have a > common doc number. Like > > > > *xml1* > *xml2* > > > > <test> > <test> > > <doc number = "*1200*"> > <doc number = "*1200*"> > > .... > ..... > > </text> > </test> > > > > - Our requirement here is that we have to insert a node is both > the documents mentioning that a document has link to another document. > Like. > > > > *xml1* > *xml2* > > > > <test> > <test> > > <doc name="refer" number = > "1200"> <doc name="refer" number = > "1200"> > > <link node > ="xml2"> > <link node = "xml1"> > > .... > ..... > > </text> > </test> > > > > *Approach* > > > > - In the directory /TEST/EMPLOYEE loop through all the document > and get all the document that has the node <doc number="xxx"> as this node > might not be present in certain documents. Then this list is sent back to > JAVA. > > > > > > for $doc in xdmp:directory($dir,"1") > > return if(doc(xdmp:node-uri($doc)) $type ) > then( > > let > $searchResponse:=search:search("", > > <options > xmlns="http://marklogic.com/appservices/search";> > > > <additional-query>{ > > > cts:and-query(( > > > cts:directory-query(($dir),"1"), > > > cts:element-attribute-value-query(xs:QName('doc'),xs:QName('name'),'refer'), > > > > ))}</additional-query>, > > > </options>)//search:result/@uri > > let $linkedDocs := if( > $searchResponse ne "" ) then ( > > > > for $lk in $searchResponse > > return > concat($document,"==>",$lk) > > ) > > else > (concat((xdmp:node-uri($doc)),"==>nolink")) > > return > $linkedDocs > > ) > > else () > > > > - This list of documents is iterated (loop) and using > search:search here, the link document ID is determined. > > - A new node is created in the corresponding xml with its linked > document name. > > > > *Problem* > > > > The problem that we are facing here is, as the document > volume is high, the first step here itself takes too long to get execute > and results in XDMP:EXTIME. To log the events we are passing the list from > Marklogic to JAVA. We already have range indexes available in these > elements. > > > > *Regards,* > > *Shaik Ummer Faruk D.* > > > > **************** CAUTION - Disclaimer ***************** > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely > for the use of the addressee(s). If you are not the intended recipient, please > notify the sender by e-mail and delete the original message. Further, you are > not > to copy, disclose, or distribute this e-mail or its contents to any other > person and > any such actions are unlawful. This e-mail may contain viruses. Infosys has > taken > every reasonable precaution to minimize this risk, but is not liable for any > damage > you may sustain as a result of any virus in this e-mail. You should carry out > your > own virus checks before opening the e-mail or attachment. Infosys reserves the > right to monitor and review the content of all messages sent to or from this > e-mail > address. Messages sent to or from this e-mail address may be stored on the > Infosys e-mail system. > ***INFOSYS******** End of Disclaimer ********INFOSYS*** > > > > _______________________________________________ > General mailing > [email protected]http://developer.marklogic.com/mailman/listinfo/general > > > > -- > > David Ennis > Content Engineer > [image: HintTech Mastering the value of content]<http://www.hinttech.com> > Mastering > the value of content creative | technology | content > Delftechpark 37i 2628 XJ Delft The Netherlands T: +31 88 268 25 00 > M: +31 6 000 000 00 > [image: Website] <http://www.hinttech.com> [image: > Twitter]<https://twitter.com/HintTech> [image: > Facebook] <http://www.facebook.com/HintTech> [image: > LinkedIn]<http://www.linkedin.com/company/HintTech> > [image: HintTech Mastering the value of content]<http://www.dayon.nl> >
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
