Re: Objects inherited in subrequests
imran wrote: Grzegorz Kossakowski wrote: Grzegorz Kossakowski pisze: It's not thread-specific but request-specific but here request means the one coming from browser and handled by servlet container and not those internal requests that Cocoon is doing when cocoon: or servlet: protocols are being used. in the setup method of the generator class we replace the objectmodel injected by spring with the one passed with that method ... after doing this the errors are not that frequent .. but they are still there :( .. its still being modified by some other threads ... Could you please let us know how do u intend to implement ur proposed solution The idea is to introduce a new implementation of a Spring scope that will work in a following way: 1. if requested object (like OM) exists in current context then just return it 2. if requested object does not yet exist in current context then find out if this context is derived from root one: a) if the context is derived then ask the root for the object, clone it and store cloned version in current context b) if the context is not derived one so itself is a root then just ask Spring to create a new instance of requested object and again store it in current context The introduction of new contexts should happen only in two cases: 1. new thread is being invoked (for multi-thread scenarios) 2. new internal request is being invoked (in case of using servlet/cocoon protocols) Even if this may sound scary, the implementation of this idea should be very simple. It's only a few lines of code are needed to handle everything. Of course, the problem is where to inject this code. Actually, today I was so busy with other work that I hadn't enough time to look into Cocoon itself but tomorrow I'll do so and give you more detailed instructions how to implement this. Hopefully, proposed solution should fix this problem once and for ever (at least I fail to imagine any situation when this wouldn't work) Imran, after taking a closer look at our code I can see that there are little bit more issues that need to be fixed before I can fix this one. I guess it's going to be much easier if I take care of a whole work because it's really non-trivial stuff and you really need to know all the internals of Cocoon in order to properly fix the problem you have. Therefore I suggest that you provide me a simple application that exhibits this problem so I can test my fixes and track the whole bug. I suggest creation of two different scenarios where one will use cocoon: protocol and another will use servlet: protocol. The most preferred way of providing me this test-cases would be to prepare two ITs as it's done in http://svn.eu.apache.org/viewvc/cocoon/trunk/blocks/cocoon-it/ http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-webapp/src/test/java/org/apache/cocoon/it/ In the first location you add sitemap entries and other resources that you usually at to your block, and in the webapp you create a test-case that tests expected behavior. As soon as you provide me such test-cases (as a patch in JIRA) I'll start working on this problem. Grzegorz, i have attached the testcase to the issue https://issues.apache.org/jira/browse/COCOON-2216 i have attached three files .. test-block is the main block .. webapp is for running that block .. before that u will have to apply the patch cocoon-trunk.patch to cocoon other wise it wont work at all ... for both the block perform mvn install .. and then from webapp block mvn jetty:run so the page http://localhost:/test-block/index.html is generated using the parallel generator .. if ur lucky and dont get the error the first time then just refresh and am sure the error will be there .. there are five components included in that page ... out of which 4 are normal jx components and the fifth one is cforms component ... all the jx component have been assigned different instance of the same object but still the properties displayed on the page shows the same value(they should be different) .. if u execute query for these components individually then they appear without any error .. links for querying them individually [jx components .. shows xml output .. forgot to add the pipeline to serialize it to html.. u can add one] http://localhost:/test-block/showjx http://localhost:/test-block/showjx1 http://localhost:/test-block/showjx2 http://localhost:/test-block/showjx3 [cforms component] http://localhost:/test-block/form2bean.flow most of the time we get the error Caused by: org.apache.commons.jxpath.JXPathException: No value for xpath: $cocoon/continuation/id its because the objectmodel is being changed by all the components simultaneously the error is there on the logs not on the page .. on the page it will just ignore that component u can change the line 427 in the class TestContentAggregator in the main block ...
Re: Objects inherited in subrequests
Grzegorz Kossakowski wrote: Grzegorz Kossakowski pisze: It's not thread-specific but request-specific but here request means the one coming from browser and handled by servlet container and not those internal requests that Cocoon is doing when cocoon: or servlet: protocols are being used. in the setup method of the generator class we replace the objectmodel injected by spring with the one passed with that method ... after doing this the errors are not that frequent .. but they are still there :( .. its still being modified by some other threads ... Could you please let us know how do u intend to implement ur proposed solution The idea is to introduce a new implementation of a Spring scope that will work in a following way: 1. if requested object (like OM) exists in current context then just return it 2. if requested object does not yet exist in current context then find out if this context is derived from root one: a) if the context is derived then ask the root for the object, clone it and store cloned version in current context b) if the context is not derived one so itself is a root then just ask Spring to create a new instance of requested object and again store it in current context The introduction of new contexts should happen only in two cases: 1. new thread is being invoked (for multi-thread scenarios) 2. new internal request is being invoked (in case of using servlet/cocoon protocols) Even if this may sound scary, the implementation of this idea should be very simple. It's only a few lines of code are needed to handle everything. Of course, the problem is where to inject this code. Actually, today I was so busy with other work that I hadn't enough time to look into Cocoon itself but tomorrow I'll do so and give you more detailed instructions how to implement this. Hopefully, proposed solution should fix this problem once and for ever (at least I fail to imagine any situation when this wouldn't work) Imran, after taking a closer look at our code I can see that there are little bit more issues that need to be fixed before I can fix this one. I guess it's going to be much easier if I take care of a whole work because it's really non-trivial stuff and you really need to know all the internals of Cocoon in order to properly fix the problem you have. Therefore I suggest that you provide me a simple application that exhibits this problem so I can test my fixes and track the whole bug. I suggest creation of two different scenarios where one will use cocoon: protocol and another will use servlet: protocol. The most preferred way of providing me this test-cases would be to prepare two ITs as it's done in http://svn.eu.apache.org/viewvc/cocoon/trunk/blocks/cocoon-it/ http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-webapp/src/test/java/org/apache/cocoon/it/ In the first location you add sitemap entries and other resources that you usually at to your block, and in the webapp you create a test-case that tests expected behavior. As soon as you provide me such test-cases (as a patch in JIRA) I'll start working on this problem. Grzegorz, i have attached the testcase to the issue https://issues.apache.org/jira/browse/COCOON-2216 i have attached three files .. test-block is the main block .. webapp is for running that block .. before that u will have to apply the patch cocoon-trunk.patch to cocoon other wise it wont work at all ... for both the block perform mvn install .. and then from webapp block mvn jetty:run so the page http://localhost:/test-block/index.html is generated using the parallel generator .. if ur lucky and dont get the error the first time then just refresh and am sure the error will be there .. there are five components included in that page ... out of which 4 are normal jx components and the fifth one is cforms component ... all the jx component have been assigned different instance of the same object but still the properties displayed on the page shows the same value(they should be different) .. if u execute query for these components individually then they appear without any error .. links for querying them individually [jx components .. shows xml output .. forgot to add the pipeline to serialize it to html.. u can add one] http://localhost:/test-block/showjx http://localhost:/test-block/showjx1 http://localhost:/test-block/showjx2 http://localhost:/test-block/showjx3 [cforms component] http://localhost:/test-block/form2bean.flow most of the time we get the error Caused by: org.apache.commons.jxpath.JXPathException: No value for xpath: $cocoon/continuation/id its because the objectmodel is being changed by all the components simultaneously the error is there on the logs not on the page .. on the page it will just ignore that component u can change the line 427 in the class TestContentAggregator in the main block ... this.parameters.setParameter(parallel,
Re: Objects inherited in subrequests
Grzegorz Kossakowski pisze: It's not thread-specific but request-specific but here request means the one coming from browser and handled by servlet container and not those internal requests that Cocoon is doing when cocoon: or servlet: protocols are being used. in the setup method of the generator class we replace the objectmodel injected by spring with the one passed with that method ... after doing this the errors are not that frequent .. but they are still there :( .. its still being modified by some other threads ... Could you please let us know how do u intend to implement ur proposed solution The idea is to introduce a new implementation of a Spring scope that will work in a following way: 1. if requested object (like OM) exists in current context then just return it 2. if requested object does not yet exist in current context then find out if this context is derived from root one: a) if the context is derived then ask the root for the object, clone it and store cloned version in current context b) if the context is not derived one so itself is a root then just ask Spring to create a new instance of requested object and again store it in current context The introduction of new contexts should happen only in two cases: 1. new thread is being invoked (for multi-thread scenarios) 2. new internal request is being invoked (in case of using servlet/cocoon protocols) Even if this may sound scary, the implementation of this idea should be very simple. It's only a few lines of code are needed to handle everything. Of course, the problem is where to inject this code. Actually, today I was so busy with other work that I hadn't enough time to look into Cocoon itself but tomorrow I'll do so and give you more detailed instructions how to implement this. Hopefully, proposed solution should fix this problem once and for ever (at least I fail to imagine any situation when this wouldn't work) Imran, after taking a closer look at our code I can see that there are little bit more issues that need to be fixed before I can fix this one. I guess it's going to be much easier if I take care of a whole work because it's really non-trivial stuff and you really need to know all the internals of Cocoon in order to properly fix the problem you have. Therefore I suggest that you provide me a simple application that exhibits this problem so I can test my fixes and track the whole bug. I suggest creation of two different scenarios where one will use cocoon: protocol and another will use servlet: protocol. The most preferred way of providing me this test-cases would be to prepare two ITs as it's done in http://svn.eu.apache.org/viewvc/cocoon/trunk/blocks/cocoon-it/ http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-webapp/src/test/java/org/apache/cocoon/it/ In the first location you add sitemap entries and other resources that you usually at to your block, and in the webapp you create a test-case that tests expected behavior. As soon as you provide me such test-cases (as a patch in JIRA) I'll start working on this problem. -- Best regards, Grzegorz Kossakowski
Re: Objects inherited in subrequests
imran pisze:
Hello all
Grzegorz as u wrote...
Therefore I would like to propose introduction of a new scope called
request-inheritable which
would allow beans (like ObjectModel) to be inherited but by cloning.
can you please elaborate on that ... coz thats where we are stuck ...
if u have a look at the AbstractInterpreter.java file
public void forwardTo(String uri, Object bizData,
WebContinuation continuation,
Redirector redirector)
throws Exception {
if (SourceUtil.indexOfSchemeColon(uri) == -1) {
uri = cocoon:/ + uri;
final Map objectModel =
this.processInfoProvider.getObjectModel();
FlowHelper.setWebContinuation(objectModel, newObjectModel,
continuation);
FlowHelper.setContextObject(objectModel, newObjectModel,
bizData);
if (redirector.hasRedirected()) {
throw new IllegalStateException(Pipeline has already
been processed for this request);
}
// this is a hint for the redirector
objectModel.put(cocoon:forward, true);
redirector.redirect(false, uri);
} else {
throw new Exception(uri is not allowed to contain a scheme
(cocoon:/ is always automatically used));
}
}
so the instance of newObjectModel is the one from the bean factory ...
and if u have multithreading then all the threads change that
objectmodel simultaneously and then finally the objectmodel coming to
the JXTemplategenerator(Injected by spring) is not thread specific ...
It's not thread-specific but request-specific but here request means the one coming from browser and
handled by servlet container and not those internal requests that Cocoon is doing when cocoon: or
servlet: protocols are being used.
in the setup method of the generator class we replace the objectmodel
injected by spring with the one passed with that method ... after doing
this the errors are not that frequent .. but they are still there :( ..
its still being modified by some other threads ...
Could you please let us know how do u intend to implement ur proposed
solution
The idea is to introduce a new implementation of a Spring scope that will work
in a following way:
1. if requested object (like OM) exists in current context then just return it
2. if requested object does not yet exist in current context then find out if this context is
derived from root one:
a) if the context is derived then ask the root for the object, clone it and store cloned version
in current context
b) if the context is not derived one so itself is a root then just ask Spring to create a new
instance of requested object and again store it in current context
The introduction of new contexts should happen only in two cases:
1. new thread is being invoked (for multi-thread scenarios)
2. new internal request is being invoked (in case of using servlet/cocoon
protocols)
Even if this may sound scary, the implementation of this idea should be very simple. It's only a few
lines of code are needed to handle everything. Of course, the problem is where to inject this code.
Actually, today I was so busy with other work that I hadn't enough time to look into Cocoon itself
but tomorrow I'll do so and give you more detailed instructions how to implement this.
Hopefully, proposed solution should fix this problem once and for ever (at least I fail to imagine
any situation when this wouldn't work)
--
Best regards,
Grzegorz Kossakowski
Re: Objects inherited in subrequests
imran pisze:
Hello all
Grzegorz as u wrote...
Therefore I would like to propose introduction of a new scope called
request-inheritable which
would allow beans (like ObjectModel) to be inherited but by cloning.
can you please elaborate on that ... coz thats where we are stuck ...
if u have a look at the AbstractInterpreter.java file
public void forwardTo(String uri, Object bizData,
WebContinuation continuation,
Redirector redirector)
throws Exception {
if (SourceUtil.indexOfSchemeColon(uri) == -1) {
uri = cocoon:/ + uri;
final Map objectModel =
this.processInfoProvider.getObjectModel();
FlowHelper.setWebContinuation(objectModel, newObjectModel,
continuation);
FlowHelper.setContextObject(objectModel, newObjectModel,
bizData);
if (redirector.hasRedirected()) {
throw new IllegalStateException(Pipeline has already
been processed for this request);
}
// this is a hint for the redirector
objectModel.put(cocoon:forward, true);
redirector.redirect(false, uri);
} else {
throw new Exception(uri is not allowed to contain a scheme
(cocoon:/ is always automatically used));
}
}
so the instance of newObjectModel is the one from the bean factory ...
and if u have multithreading then all the threads change that
objectmodel simultaneously and then finally the objectmodel coming to
the JXTemplategenerator(Injected by spring) is not thread specific ...
in the setup method of the generator class we replace the objectmodel
injected by spring with the one passed with that method ... after doing
this the errors are not that frequent .. but they are still there :( ..
its still being modified by some other threads ...
Could you please let us know how do u intend to implement ur proposed
solution
Just wanted to let you know that I know this stuff really well (I've worked on ObjectModel last
summer) so surely can give you some tips and precise instructions.
This won't happen now as I'm leaving but I'll try to explain my thoughts tomorrow after I check
everything against Cocoon's source code.
--
Best regards,
Grzegorz Kossakowski
Re: Objects inherited in subrequests
sorry, was to quick :(( So I decided to patch the DefaultIncludeCacheManager to work in the same way as the IncludeTransformer does, by setting up the child threads with the same context/environment: // Setup this thread's environment RequestContextHolder.setRequestAttributes( this.session.getRequestAttributes()); EnvironmentHelper.enterProcessor(this.session.getProcessor(), this.session.getEnvironment()); I'll test this out, and if it work's I send in the patch. it does not help. It's the way it's meant to work, but it doesn't. Somehow the ObjectModels in the parallel running JXTemplateGenerators affect each other and we have strange side affects on most of our request. we are digging around, but no ideas any more how to fix it... any help would be appreciated. Christoph
Re: Objects inherited in subrequests
Hello all
Grzegorz as u wrote...
Therefore I would like to propose introduction of a new scope called
request-inheritable which
would allow beans (like ObjectModel) to be inherited but by cloning.
can you please elaborate on that ... coz thats where we are stuck ...
if u have a look at the AbstractInterpreter.java file
public void forwardTo(String uri, Object bizData,
WebContinuation continuation,
Redirector redirector)
throws Exception {
if (SourceUtil.indexOfSchemeColon(uri) == -1) {
uri = cocoon:/ + uri;
final Map objectModel =
this.processInfoProvider.getObjectModel();
FlowHelper.setWebContinuation(objectModel, newObjectModel,
continuation);
FlowHelper.setContextObject(objectModel, newObjectModel,
bizData);
if (redirector.hasRedirected()) {
throw new IllegalStateException(Pipeline has already
been processed for this request);
}
// this is a hint for the redirector
objectModel.put(cocoon:forward, true);
redirector.redirect(false, uri);
} else {
throw new Exception(uri is not allowed to contain a scheme
(cocoon:/ is always automatically used));
}
}
so the instance of newObjectModel is the one from the bean factory ...
and if u have multithreading then all the threads change that
objectmodel simultaneously and then finally the objectmodel coming to
the JXTemplategenerator(Injected by spring) is not thread specific ...
in the setup method of the generator class we replace the objectmodel
injected by spring with the one passed with that method ... after doing
this the errors are not that frequent .. but they are still there :( ..
its still being modified by some other threads ...
Could you please let us know how do u intend to implement ur proposed
solution
regards
Imran
Christoph Gaffga (triplemind.com) wrote:
sorry, was to quick :((
So I decided to patch the DefaultIncludeCacheManager to work in the
same
way as the IncludeTransformer does, by setting up the child threads
with
the same context/environment:
// Setup this thread's environment
RequestContextHolder.setRequestAttributes(
this.session.getRequestAttributes());
EnvironmentHelper.enterProcessor(this.session.getProcessor(),
this.session.getEnvironment());
I'll test this out, and if it work's I send in the patch.
it does not help. It's the way it's meant to work, but it doesn't.
Somehow the ObjectModels in the parallel running JXTemplateGenerators
affect each other and we have strange side affects on most of our
request.
we are digging around, but no ideas any more how to fix it...
any help would be appreciated.
Christoph
Re: Objects inherited in subrequests
Grzegorz Kossakowski wrote: Therefore I would like to propose introduction of a new scope called request-inheritable which would allow beans (like ObjectModel) to be inherited but by cloning. What do you think? hmm, just tried to solve that problem and found that in IncludeTransformer it works with threading and parallel processing. Only for the DefaultIncludeCacheManager we are using in our replacement for the aggregator to do parallel aggregation it is broken since version 2.2. So I decided to patch the DefaultIncludeCacheManager to work in the same way as the IncludeTransformer does, by setting up the child threads with the same context/environment: // Setup this thread's environment RequestContextHolder.setRequestAttributes( this.session.getRequestAttributes()); EnvironmentHelper.enterProcessor(this.session.getProcessor(), this.session.getEnvironment()); I'll test this out, and if it work's I send in the patch. Christoph
Objects inherited in subrequests (was: Re: multithreaded Content Aggregator migration from 2.1 to 2.2)
Joerg Heinicke pisze:
On 06.06.2008 12:51, Grzegorz Kossakowski wrote:
E.g. if two threads share the same (inherited) instance of ObjectModel
all nasty side-effects of multi-threading should be expected like
parallel modification and loosing of data consistency kept in OM.
How Cocoon used to solve these kind of problems in the past?
I don't see that much of a problem here - and it always used to be that
way. If it is a distinctive object model or just the request object does
not really make a difference to me. Also in other frameworks where you
only work with request or session objects you have to be aware of
multi-threading.
I agree in general but there is a subtle difference here. I think there are plenty of people that
agree on a statement that one doesn't need to be a Java programmer in order to use Cocoon. There are
many success stories proving that.
In Cocoon you can have complicated scenarios that you introduce only by using standard Cocoon
components and pipelines all glued by sitemap. In such scenarios, when one does not write any line
of custom java code she should be protected from tricky troubles like the one Imran ran into.
The whole issue comes down to finding an answer to crucial question that appeared here already many
times:
How much do we want to inherit from original request (more generally
environment) to subrequests?
The ideal situation would be if we could inherit nothing thus avoiding any tricky problems but such
solution is rather unacceptable because it brings another sort of problems.
Another idea I can think of is to have a clearly and narrowly defined set of things (objects, data)
we want to inherit in subrequests and pay as much attention as we can to assuring that these objects
are protected from any side-effects including running in multi-threaded environments.
At the moment I can think only about two kind of data that we should inherit,
it is:
a) request data (uploaded files, attributes, etc.)
b) ObjectModel (the one defined in cocoon-expression-api that stores for example data coming from
flowscript)
I think the only way we can avoid side-effects is by _cloning_ objects that are inherited so
subrequests do not affect each other and original request (and their environments). Actually, that
not a new idea, see EnvironmentHelper.AbstractCocoonRunnable[1]:
public AbstractCocoonRunnable() {
// Clone the environment stack of the calling thread.
// We'll use it in run() below
Object stack = EnvironmentHelper.environmentStack.get();
if (stack != null) {
this.parentStack = ((EnvironmentStack)stack).clone();
}
}
Therefore I would like to propose introduction of a new scope called request-inheritable which
would allow beans (like ObjectModel) to be inherited but by cloning.
What do you think?
Is the multi-threading processing a feature one has to switch on
intentionally?
Yes, but even if you enable it intentionally you still don't have be a Java programmer so all above
applies here as well.
[1]
http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-sitemap/cocoon-sitemap-impl/src/main/java/org/apache/cocoon/environment/internal/EnvironmentHelper.java?view=markup
--
Grzegorz Kossakowski
