I'm surprised that the for loop doesn't run xdmp:http-post in parallel. If
you really need the looping and parallel posting capability you could do
something like this:
let $endpoints := element endpoints {
element endpoint {"http://localhost:8469/"},
element endpoint {"http://localhost:8469/"},
element endpoint {"http://localhost:8469/"},
element endpoint {"http://localhost:8469/"},
element endpoint {"http://localhost:8469/"}
}
let $range := 1 to count($endpoints/endpoint)
return
xdmp:eval(concat("
declare variable $endpoints as element(endpoints) external;
let ",string-join(for $i in $range
return concat("$post",$i," :=
xdmp:http-post($endpoints/endpoint[",$i,"])"),','),
" return (", string-join(for $i in $range return
concat("$post",$i),','),")")
,(fn:QName("","endpoints"),$endpoints))
Normally I would shy away from using xdmp:eval, but in this case the code
is tight enough that it doesn't open holes for XQuery injection (unless I'm
missing something, feel free to point it out) and you get the added
performance of running the posts in parallel, which can make a big
difference.
-Ryan Dew
On Wed, May 23, 2012 at 10:58 AM, [email protected]
<[email protected]>wrote:
> I have a search function that calls other search functions that are
> independent of each other. The master search function takes the results of
> these sub-search functions and compiles and processes them according to
> some rules.
>
> I would like the sub-search functions to all execute in parallel at the
> same time to reduce the response time of the call to the master search
> function.
>
> I tried using xdmp:http-post and xdmp:invoke both in for loops and also
> without for loops.
>
> for example:
>
> *xdmp:http-post (for loop) does not execute in parallel:
>
> let $sub-search-results :=
> for $sub-search in $sub-searches
> return xdmp:http-post($endpoint....)
>
> let $subsearch-results :=
> <subsearch-results>
> {$sub-search-results}
> </subsearch-results>
>
> return $sub-search-results
>
>
> *xdmp:http-post (no for loop) DOES execute in parallel:
>
> let $sub-search-result-1 := xdmp:http-post($endpoint....)
> let $sub-search-result-2 := xdmp:http-post($endpoint....)
> let $sub-search-result-3 := xdmp:http-post($endpoint....)
> let $sub-search-result-4 := xdmp:http-post($endpoint....)
>
> let $subsearch-results :=
> <subsearch-results>
> {$sub-search-result-1}
> {$sub-search-result-2}
> {$sub-search-result-3}
> {$sub-search-result-4}
> </subsearch-results>
>
> return $sub-search-results
>
>
> I tried similar approaches using xdmp:invoke (both with and without a for
> loop) but neither would execute in parallel.
>
> Does this sound right? I was expecting to be able to get parallel
> execution in a for loop using xdmp:http-post (I even tried fn:unordered but
> it had not effect). and I was expecting to get parallel execution using
> xdmp:invoke somehow but I couldn't.
>
> The http-post approach isn't too bad for my situation but it feels like a
> little bit of a hack just to get parallel execution. Are there any other
> ways?
>
> thanks,
> Ryan
>
> _______________________________________________
> General mailing list
> [email protected]
> http://community.marklogic.com/mailman/listinfo/general
>
>
_______________________________________________
General mailing list
[email protected]
http://community.marklogic.com/mailman/listinfo/general
