Also, because the cts:search is in a for loop and building up a result set, the entire cts:search result set must be put in memory in order to build your result set. If you just returned the cts:search, it would still be a lot of results, but it could stream it out. Like Geert says, pagination is a great way to do these kinds of things.
-Danny From: [email protected] [mailto:[email protected]] On Behalf Of Gary Larsen Sent: Thursday, August 11, 2011 2:02 PM To: 'General MarkLogic Developer Discussion' Subject: Re: [MarkLogic Dev General] Quering documents with fragments HI Geert, Thanks for clarifying that only query results are stored to the cache, and I take from your suggestion of paging that read only query results would not clog the cache. I may have some unexpected data somewhere and need to research that. Thanks! gary From: [email protected] [mailto:[email protected]] On Behalf Of Geert Josten Sent: Thursday, August 11, 2011 4:38 PM To: General MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Quering documents with fragments Hi Gary, The cts querying part is based on indexes only, so no fragments nor data itself is involved. It is the compilation of the results structure that is causing the problem. To put it simple: you are returning way too many results at once. Start using pagination. Try returning something like 20 or 100 results per page. You will see that the pagination works really fast. You might also want to take a look at search:search, it can do pagination for you as well. Use additional-query for the directory-query, and perhaps a constraint for your keys (so you could use "key:mykey1").. Kind regards, Geert Van: [email protected] [mailto:[email protected]] Namens Gary Larsen Verzonden: donderdag 11 augustus 2011 21:18 Aan: 'General MarkLogic Developer Discussion' Onderwerp: [MarkLogic Dev General] Quering documents with fragments Hi, I try solve a query which is throwing XDMP-EXPNTREECACHEFULL errors. It's possible that the documents being queried have a large number of fragments (> 30K). The query is not referencing any elements in the fragments, but I'm wondering if the query is loading the root document AND fragments into the expanded tree cache. Here's main part of the query let $cq := cts:and-query(( cts:directory-query('/db/netvisn/content/','infinity'), cts:element-value-query(xs:QName('key'), $keys) )) return <results> {for $info in cts:search(doc(), $cq, 'filtered')/content/lookupInfo return <SyncData>{$info/key} Do I need to surround the cts:element-value-query() with a cts:document-fragment-query() to avoid grabbing the fragments? Thanks
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
