Thanks once more, Geert! >From an architectural point of view, does it make sense to have a loop over many thousand URLs run in an xquery which may take 5 seconds each? How does Mark Logic handle long-running queries? The goal is to assemble the information about the accessibility of these URLs in an XML document that will be stored in Mark Logic and used for analytical output. Should the list be fragmented into smaller sub-lists and be processed separately?
Would it be possible to have several threads run simultaneously? Somehow I doubt it as there would probably be issues with the final aggregation of the different thread sub-documents into a big one. What about timeouts, especially if the function is called from inside a web page, how does Mark Logic handle this issue (I saw that one can tweak the timeout), or would this be a browser timeout problem? I hope these questions are not too off-topic for the list. Jakob. On Thu, Jun 25, 2009 at 08:11, Geert Josten <[email protected]> wrote: > > Hi Jakob, > > You are looking for unbuffered response streams, but sending of the response > is handled fully by the HTTP server. I don't believe you can influence that. > > Giving it some more thought, I am afraid that allowing unbuffered responses > would break the idea of transactions. You don't want to send back response, > unless you can guarantee no exceptions will be thrown. And I don't think that > can be guaranteed. > > Perhaps a MarkLogic expert would like to comment? > > Hasn't this been discussed before? It vaguely rings a bell.. > > Kind regards, > Geert > > > > > > Drs. G.P.H. Josten > Consultant > > > http://www.daidalos.nl/ > Daidalos BV > Source of Innovation > Hoekeindsehof 1-4 > 2665 JZ Bleiswijk > Tel.: +31 (0) 10 850 1200 > Fax: +31 (0) 10 850 1199 > http://www.daidalos.nl/ > KvK 27164984 > De informatie - verzonden in of met dit emailbericht - is afkomstig van > Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit > bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit > bericht kunnen geen rechten worden ontleend. > > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of > > Jakob Fix > > Sent: donderdag 25 juni 2009 1:39 > > To: General Mark Logic Developer Discussion > > Subject: [MarkLogic Dev General] force a "buffer write"? > > > > So, I've written a function that looks at one URL at a time > > and returns true or false depending on its accessibility. > > Now, my problem is that the result is returned only when all > > (and that means potentially many) URLs have been checked. > > Isn't there a way to "force a write"? I'm not sure I'm > > expressing myself correctly, but hopefully you'll understand > > what I mean. Thanks. > > > > (: consider this: if the timeout is 10 seconds, and I have > > three "bad" URLs, I may have to wait up to 30 seconds before > > seeing the results, isn't there a way to see a result every > > ten seconds instead? :) > > > > for $url in $urls > > return <xh:li>{$url}: > > {utils:http-resource-available($url, $timeout)}</xh:li> > > > > > > (: function that checks accessibility of a URL, written for > > DOIs which are "placeholder" URLs which forward to the real URL :) > > > > declare function utils:http-resource-available > > ($doi as xs:string, $oldtimeout as xs:integer?) as xs:boolean { > > try { > > > > let $timeout := if ($oldtimeout) then $oldtimeout else 10 > > > > let $head := > > xdmp:http-get(fn:concat($utils:doi-resolver, $doi), > > <options > > xmlns="xdmp:http"><timeout>{$timeout}</timeout></options>) > > let $code := $head//xdh:code cast as xs:integer > > let $location := $head//xdh:location > > return ((fn:contains($location, > > $utils:location-part-to-match)) and > > ($code < 400)) (: we want 3XX or 2XX? :) > > } catch ($ex) { > > if ($ex/error:code eq 'SVC-SOCRECV') > > then > > fn:false() > > else > > xdmp:rethrow() > > } > > }; > > > > > > > > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
