Hi Danny,
I didn't realize side effect of the for loop. Below is the original query
(converted from eXist) where I'm trying to create a structure that's passed
to a Stax handler in Java. Perhaps a better approach would be to run the
$cq and cts:search for each key rather than all at once.
$keys := ("i9297889D3DB8430DB22F66EF3BA331DF",
"i955A9E9DE3964B3A91D4F26BE35C88EB", "i598A118E534B4A5DA42FDBB36FF039AA",
"i7A8EAC0921E049D98CC80C3A6DC032A7", "i8DB23187DFCD4D95AF094A2DDB7F43EA")
let $cq := cts:and-query((cts:directory-query("/db/netvisn/content/",
"infinity"),
cts:element-value-query(fn:QName( "key"), $keys)))
return <results>
{ for $info in cts:search(fn:doc(), $cq, "filtered")/content/lookupInfo
return
<SyncData>{ $info/key }{ $info }</SyncData> }
</results>
Thanks,
Gary
From: [email protected]
[mailto:[email protected]] On Behalf Of Danny Sokolsky
Sent: Thursday, August 11, 2011 5:07 PM
To: General MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Quering documents with fragments
Also, because the cts:search is in a for loop and building up a result set,
the entire cts:search result set must be put in memory in order to build
your result set. If you just returned the cts:search, it would still be a
lot of results, but it could stream it out. Like Geert says, pagination is
a great way to do these kinds of things.
-Danny
From: [email protected]
[mailto:[email protected]] On Behalf Of Gary Larsen
Sent: Thursday, August 11, 2011 2:02 PM
To: 'General MarkLogic Developer Discussion'
Subject: Re: [MarkLogic Dev General] Quering documents with fragments
HI Geert,
Thanks for clarifying that only query results are stored to the cache, and I
take from your suggestion of paging that read only query results would not
clog the cache.
I may have some unexpected data somewhere and need to research that.
Thanks!
gary
From: [email protected]
[mailto:[email protected]] On Behalf Of Geert Josten
Sent: Thursday, August 11, 2011 4:38 PM
To: General MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Quering documents with fragments
Hi Gary,
The cts querying part is based on indexes only, so no fragments nor data
itself is involved. It is the compilation of the results structure that is
causing the problem. To put it simple: you are returning way too many
results at once. Start using pagination. Try returning something like 20 or
100 results per page. You will see that the pagination works really fast.
You might also want to take a look at search:search, it can do pagination
for you as well. Use additional-query for the directory-query, and perhaps a
constraint for your keys (so you could use "key:mykey1")..
Kind regards,
Geert
Van: [email protected]
[mailto:[email protected]] Namens Gary Larsen
Verzonden: donderdag 11 augustus 2011 21:18
Aan: 'General MarkLogic Developer Discussion'
Onderwerp: [MarkLogic Dev General] Quering documents with fragments
Hi,
I try solve a query which is throwing XDMP-EXPNTREECACHEFULL errors. It's
possible that the documents being queried have a large number of fragments
(> 30K). The query is not referencing any elements in the fragments, but
I'm wondering if the query is loading the root document AND fragments into
the expanded tree cache. Here's main part of the query
let $cq := cts:and-query((
cts:directory-query('/db/netvisn/content/','infinity'),
cts:element-value-query(xs:QName('key'), $keys)
)) return <results>
{for $info in cts:search(doc(), $cq, 'filtered')/content/lookupInfo return
<SyncData>{$info/key}
Do I need to surround the cts:element-value-query() with a
cts:document-fragment-query() to avoid grabbing the fragments?
Thanks
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
