A lot of help here. Thanks all. Gary
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Ron Hitchens Sent: Tuesday, March 20, 2012 5:33 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Java XCC save query result to file While Mike is right (he always is, basically), the XQuery-side streaming he's describing is slightly different than XCC's definition of streaming. By default, XCC issues your request to MarkLogic then reads the entire result into memory before returning from the submitRequest() method. By the time it returns, it has released the connection and MarkLogic is done with you. The entire result is cached in Java memory. This is XCC's "cache mode" - not streaming - and it can blow memory if your XQuery returns too much data. But because it's all buffered, it means you can access ResultItems randomly and repeatedly in the ResultSequence. Cached mode is more efficient and is appropriate for the common use case of fetching results to service a web request. You can set an option on your XCC request to specify that you want "non-cached mode" - streaming. In that case, XCC does not automatically read the full response. It waits for you to fetch the items from the ResultSequence and reads the response from MarkLogic as needed. If you read some/write some, then you can pass large amounts of data through without holding it all locally. Even better is to use the writeTo() method which will do that for you. But if you're streaming and don't fetch the result items in a timely manner, MarkLogic may hang up the connection on you even if the query result is not especially large. It's also important to close streaming ResultSequences after use, since they are holding the underlying connection which can't be re-used until you do. I guess I still remember all that stuff. On Mar 20, 2012, at 9:01 PM, Michael Blakeley wrote: > Whether or not a result is capable of streaming will depend on the XQuery too. I just finishing writing something about this: > > http://blakeley.com/blogofile/2012/03/19/let-free-style-and-streaming/ > > -- Mike > > On 20 Mar 2012, at 13:58 , David Lee wrote: > >> How critical to your app is it that the result is "streamed" ? >> Is it always XML or is it sometimes text or binary ? >> >> The xmlsh MarkLogic extension "get" command can be used as an example of writing the result items to a Java stream. >> >> The "writeResult" method implements the guts of this command >> >> http://xmlsh.svn.sourceforge.net/viewvc/xmlsh/extensions/marklogic/src/org/x mlsh/marklogic/util/MLCommand.java?revision=671&view=markup >> >> The key is that a ResultSequence is an iterator of XdmItem. >> Each XdmItem has a "writeTo" method. >> >> There is no guarentee that this is "streamed" to my knowledge but it may well be. >> >> >> >> >> >> >> ---------------------------------------------------------------------------- - >> David Lee >> Lead Engineer >> MarkLogic Corporation >> [email protected] >> Phone: +1 650-287-2531 >> Cell: +1 812-630-7622 >> www.marklogic.com >> >> This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. >> >> From: [email protected] [mailto:[email protected]] On Behalf Of Gary Larsen >> Sent: Tuesday, March 20, 2012 4:38 PM >> To: 'General MarkLogic Developer Discussion' >> Subject: [MarkLogic Dev General] Java XCC save query result to file >> >> Hi, >> >> For some queries I'm using the xdmp:save() function to store results to file. This will not always work because the MarkLogic server may not have access to the share location. >> >> Instead the results will need to be saved by the application by streaming the ResultSequence returned from session.submitRequest(request) call. >> >> Is there a preferred way to do this? >> >> Thanks for your advice. An example would be helpful if you have one. >> >> Gary >> >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general --- Ron Hitchens {mailto:[email protected]} Ronsoft Technologies +44 7879 358 212 (voice) http://www.ronsoft.com +1 707 924 3878 (fax) Bit Twiddling At Its Finest "No amount of belief establishes any fact." -Unknown _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
