Hello all,
Digged a little further to find out why external URLs inclusion cannot be done
in parallel.
It seems like the problem is located in the
org.apache.cocoon.transformation.IncludeTransformer line 563 (Cocoon 2.1.11) :
synchronized (validity) {
validity.addSource(source);
}
The second "external" inclusion processed by the IncludeTransformer is waiting
for synch on the addSource method call.
When working with sitemap inclusions (cocoon://), the source.getValidity
performed by the addSource method returns immediately without really invoking
the URL. There is no synchronization issue here.
When working with external URL inclusions (http://), the source.getValidity
makes the remote call and waits for the element to be generated, thus creating
synchronization issues preventing the use of parallel inclusion of external
URL. Each inclusion will wait on the synchronize until the previous call
returns.
Is there a way to workaround this ?
Thank you,
Christophe
From: Christophe Fondacci
Sent: Thursday, June 11, 2009 11:03 AM
To: [email protected]
Subject: Re: IncludeTransformer: 2 parallel threads only ?
Update :
Default parallel inclusion seems to work fine with any number of threads.
To workaround the last point of my previous email (parallel inclusion of
"external" URLs), I made a XSP wrapper which is a XSP to include locally
(through "cocoon:/"), though enabling parallelism, which simply includes
external URL.
When I use this XSP to wrap all my external inclusions, parallelism do work but
limits the parallelism to 2 threads.
Here is an extract of the content of this wrapper XSP :
<content>
<xsp:logic>
final String myURL = <util:get-sitemap-parameter name="my_url"/>;
final String xspRef = request.getParameter(CocoonURLUtils.PARAM_XSP_REF);
final String urlQuery = CocoonURLUtils.getURLQuery(request);
<i:include>
<xsp:attribute
name="src"><xsp:expr>myURL</xsp:expr>/<xsp:expr>xspRef</xsp:expr><xsp:expr>urlQuery</xsp:expr></xsp:attribute>
</i:include>
</xsp:logic>
</content>
Thanks to anyone who could explain me why do i have this behaviour.
Christophe.
From: Christophe Fondacci
Sent: Thursday, June 11, 2009 10:26 AM
To: [email protected]
Subject: IncludeTransformer: 2 parallel threads only ?
Hello all,
I am currently working with the parallel inclusion through the
IncludeTransformer. Our website is based on Cocoon 2.1.11.
It works fine and we truly like this feature, but I notice some strange
behaviour :
a.. I have a XSP page with 4 "i:include" inclusions. We configured the
include transformer in the sitemap to enable parallelism.
b.. I placed breakpoints on every included page to see how cocoon processes
the inclusions.
c.. When I try to generate this page, it seems like inclusions are made 2 by
2. I see 2 of my breakpoints being reached, the others being reached only after
completion of these first inclusions. The weird thing is that the order seem to
change randomly : sometime i got inclusion 1 and 4 first, sometime 1 and 2,
etc. I would have expected to see my 4 breakpoints being reached in 4 distinct
threads...
Could someone explain to me if it is possible to include these 4 elements in 4
parallel threads or whether there are some limitations or restrictions on the
parallelism ?
The other thing (less important since i found a workaround) is that parallelism
seem to only work when referencing local elements through the cocoon protocol
(URI like "cocoon:/"). As soon as we reference inclusions through "external"
URLs (URI like "http://";), every inclusion seem to be processed sequentially.
Could you confirm this is the expected behaviour ? If someone can explain me
why it is working this way I would be very interested.
Thank your very much for your help / time
& keep up the good work !
Christophe.
