Re: [C3] Concurrency issues with ComponentProvider
Hey guys, has there been any progress in this matter? Any new insights? We have a jira issue classified as Major Bug accusing us of concurrency issues - IMO the worst thing that can happen to a web application framework - and there's been no word for almost 2 weeks. I'd like to see that resolved ASAP. Anything I can do to help? Steven Am 17.03.2012 03:25, schrieb Thorsten Scherler: On 03/16/2012 02:33 PM, Javier Puerto wrote: ... Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it. I am ATM creating a static domain in the httpd frontend and I observed that with the dojo block we reach 100 requests for the homepage, so I suspect that dojo maybe *the* component that can produce the problems. ...need to finish up now that is why I am so brief. salu2 -- Thorsten Scherlerscherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/
Re: [C3] Concurrency issues with ComponentProvider
On 03/29/2012 12:15 PM, Steven Dolg wrote: Hey guys, has there been any progress in this matter? Any new insights? Sorry last stand was that Javier is working on a block to reproduce the issue outside our production env but he is ATM bombed with other tasks since we entered the final testing phase of the app. The last thing he showed me was that in the rampup phase the first request were always wrong, but he may be more specific on that. We have a jira issue classified as Major Bug accusing us of concurrency issues - IMO the worst thing that can happen to a web application framework - and there's been no word for almost 2 weeks. Due to the patch we applied it is not directly open but I share your point we should get to the bottom of the issue. I'd like to see that resolved ASAP. Anything I can do to help? @Javier can you mockup something we have and state the steps we need to reproduce, so Steve may see the problem right away. salu2 Steven Am 17.03.2012 03:25, schrieb Thorsten Scherler: On 03/16/2012 02:33 PM, Javier Puerto wrote: ... Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it. I am ATM creating a static domain in the httpd frontend and I observed that with the dojo block we reach 100 requests for the homepage, so I suspect that dojo maybe *the* component that can produce the problems. ...need to finish up now that is why I am so brief. salu2 -- Thorsten Scherlerscherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/ -- Thorsten Scherlerscherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/
Re: [C3] Concurrency issues with ComponentProvider
El 29 de marzo de 2012 18:01, Thorsten Scherler scher...@gmail.comescribió: On 03/29/2012 12:15 PM, Steven Dolg wrote: Hey guys, has there been any progress in this matter? Any new insights? Sorry last stand was that Javier is working on a block to reproduce the issue outside our production env but he is ATM bombed with other tasks since we entered the final testing phase of the app. The last thing he showed me was that in the rampup phase the first request were always wrong, but he may be more specific on that. Sorry for the delay. I was be able to reproduce it consistently with a JMeter stress test for our web application (resource switch), for the concurrency block attached in the JIRA issue the error is similar but not identical (0 lenght content). With the synchronizes the problems can't be reproduced. I just request static resources served with FileReader component, 30 thread, 1s for the rampup and random request. All requests with size assertion. To reproduce the error, run the concurrency block and then the JMeter script. The first requests returns a 200 code and an empty response (image switch in our application). We have a jira issue classified as Major Bug accusing us of concurrency issues - IMO the worst thing that can happen to a web application framework - and there's been no word for almost 2 weeks. Due to the patch we applied it is not directly open but I share your point we should get to the bottom of the issue. I'd like to see that resolved ASAP. Anything I can do to help? @Javier can you mockup something we have and state the steps we need to reproduce, so Steve may see the problem right away. I'm don't know what could be the cause, we have too much components involved: I18nTransformer, Include, RegexLinkRewriterTransformer, Shiro block, Rest block, StringTemplateGenerator, XsltTransformer, servlet-service components, etc ... The test attached is how far as I've come. Salu2 salu2 Steven Am 17.03.2012 03:25, schrieb Thorsten Scherler: On 03/16/2012 02:33 PM, Javier Puerto wrote: ... Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it. I am ATM creating a static domain in the httpd frontend and I observed that with the dojo block we reach 100 requests for the homepage, so I suspect that dojo maybe *the* component that can produce the problems. ...need to finish up now that is why I am so brief. salu2 -- Thorsten Scherler scherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/ -- Thorsten Scherler scherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/
Re: [C3] Concurrency issues with ComponentProvider
Hello again, using the test project from Thorsten [1] I created a very simple client, running several threads to send requests for a randomly selected static resource (one of the image files). (Of course I reverted the synchronized in SpringComponentProvider before running the tests!) I ran several scenarios, the biggest one was 1,000,000 requests with 100 concurrent threads. After each request, the size of the response was compared to the actual size of the requested file. I ran this several times, both with and without caching, and didn't get a single error. The system handled nearly 1,000 requests per second, so I don't think it's not enough load. I could probably push it a little more, but I doubt this would show a different result. This leads me to believe that there's nothing wrong with handling concurrent requests, even under heavy load, when handling static resources or other simple pipelines. (Which is also consistent with our Jira, which is not crawling with random strangeness issues) Since you *do* have an issue, we should look for other potential sources. Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. What is your controller doing and could you replace it with some kind of static resource to test if that's causing your problem? Could you try to remove the synchronized modifiers one by one and see which of them is actually fixing your problem? (I guess it's the one at createComponent, but who knows...). Could you verify that the HTML page generated by your controller (that's how I understood your application) is actually correct when you see a wrong result? The other thing I noticed yesterday, is that the synchronized had less a performance impact than I expected. The number of requests handled per second was almost unchanged, probably due to the method being synchronized having such a short execution time. Still I think we should try to avoid this, if at all possible, since we are forcing the creating of pipelines to be handled sequentially. Steven [1] https://issues.apache.org/jira/browse/COCOON3-93 Am 14.03.2012 13:55, schrieb Javier Puerto: Hi Cocoon developers, we are working in a project based on C3 and we found a strange behaviour when loading the static resources. We are developing a Web 2.0 application and therefore we have a lot of resources (js, images and css). The weird thing is that sometimes the system is returning the wrong resources, returning a js instead of a css or switching images... For my tests I did a little modification in class FileReaderComponent to see the source of the file that is being served. See attached file FileReaderComponent.diff. This modification uses System.out to show the source of the file before and after read it. The component seems to be fine, also it's very simple so it seems to not be the cause. The patch allow me to discard the component as cause of problem, the source passed as argument when there's too much resources requests are wrong sometimes. I've review how the components are instantiated and I saw a possible design problem for ComponentProvider class. bean name=org.apache.cocoon.sitemap.ComponentProvider class=org.apache.cocoon.sitemap.SpringComponentProvider property name=actionFactory ref=org.apache.cocoon.sitemap.spring.ActionFactory / property name=languageInterpreterFactory ref=org.apache.cocoon.sitemap.expression.LanguageInterpreterFactory / property name=pipelineComponentFactory ref=org.apache.cocoon.sitemap.spring.PipelineComponentFactory / property name=pipelineFactory ref=org.apache.cocoon.sitemap.spring.PipelineFactory / /bean bean name=org.apache.cocoon.sitemap.Invocation class=org.apache.cocoon.sitemap.InvocationImpl scope=prototype property name=componentProvider ref=org.apache.cocoon.sitemap.ComponentProvider / /bean Above is the sitemap component configuration, the InvocationImpl is the component that will create the different components thought the ComponentProvider that will delegate on different factories. The problem I see is that while the Invocation is declared as prototype, the ComponentProvider is a shared instance so it could be that on high demand the methods overlaps between request. I've solved the problem declaring the methods as synchronized (att. SpringComponentProvider.diff). This will solve the problems with concurrency, but I wonder if it's the best solution. WDYT? It's very hard to reproduce the problem with tools like JMeter (better reloading directly in browser). Anyways it's happend sometimes with my current functional test plan but I can't reproduce it consistently. Also the JMeter test plan is designed for our current web
Re: [C3] Concurrency issues with ComponentProvider
El 16 de marzo de 2012 11:07, Steven Dolg steven.d...@indoqa.com escribió: Hello again, using the test project from Thorsten [1] I created a very simple client, running several threads to send requests for a randomly selected static resource (one of the image files). I was working on it yesterday too. :) (Of course I reverted the synchronized in SpringComponentProvider before running the tests!) Be careful if you have the snapshots maven repository configured, I had to run my test in offline mode. I ran several scenarios, the biggest one was 1,000,000 requests with 100 concurrent threads. After each request, the size of the response was compared to the actual size of the requested file. I ran this several times, both with and without caching, and didn't get a single error. The system handled nearly 1,000 requests per second, so I don't think it's not enough load. I could probably push it a little more, but I doubt this would show a different result. This leads me to believe that there's nothing wrong with handling concurrent requests, even under heavy load, when handling static resources or other simple pipelines. I got the same conclusions. For the tests, I scraped some pages from internet put them on COB-INF and serve them with a FileReader component. My JMeter stress test didn't reproduced the problem. (Which is also consistent with our Jira, which is not crawling with random strangeness issues) Since you *do* have an issue, we should look for other potential sources. +1 Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it. What is your controller doing and could you replace it with some kind of static resource to test if that's causing your problem? Could you try to remove the synchronized modifiers one by one and see which of them is actually fixing your problem? (I guess it's the one at createComponent, but who knows...). Yeah, I added synchronized to all the methods but I think the same, probably createComponent is enough. Could you verify that the HTML page generated by your controller (that's how I understood your application) is actually correct when you see a wrong result? It was my first test with JMeter and seems to work fine, also when I saw the problem with the FileReader patch, I inspected the HTML and the problematic file source was right. If I request the file again the image is fine. The other thing I noticed yesterday, is that the synchronized had less a performance impact than I expected. The number of requests handled per second was almost unchanged, probably due to the method being synchronized having such a short execution time. I think that the syncs are only to retrieve the component instances so it should be a lightweight operation. Also the process can continue to be concurrently, only the instances are retrieved sequentially. So the performance impact will be bigger for component with higher creation costs. Maybe I'm wrong. Still I think we should try to avoid this, if at all possible, since we are forcing the creating of pipelines to be handled sequentially. I'm agree, it should be better to fix the problematic component but the patch will fix the problem until we found the problematic component. For more testing I was trying to make the profiling block to work without luck. Does somebody tried to profile a C3 application yet? I didn't found the documentation, I was guiding myself with the source code. Steven Thank Steven for your tests and efforts. Salu2. [1] https://issues.apache.org/**jira/browse/COCOON3-93https://issues.apache.org/jira/browse/COCOON3-93 Am 14.03.2012 13:55, schrieb Javier Puerto: Hi Cocoon developers, we are working in a project based on C3 and we found a strange behaviour when loading the static resources. We are developing a Web 2.0 application and therefore we have a lot of resources (js, images and css). The weird thing is that sometimes the system is returning the wrong resources, returning a js instead of a css or switching images... For my tests I did a little modification in class FileReaderComponent to see the source of the file that is being served. See attached file FileReaderComponent.diff. This modification uses System.out to show the source of the file before and after read it. The component seems to be fine, also it's very simple so it seems to not be the cause. The patch allow me to discard the component as cause of problem,
Re: [C3] Concurrency issues with ComponentProvider
On 03/16/2012 02:33 PM, Javier Puerto wrote: ... Controllers in general (or your specific one) could be causing problems. (The synchronized in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday. We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it. I am ATM creating a static domain in the httpd frontend and I observed that with the dojo block we reach 100 requests for the homepage, so I suspect that dojo maybe *the* component that can produce the problems. ...need to finish up now that is why I am so brief. salu2 -- Thorsten Scherlerscherler.at.gmail.com codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/
Re: [C3] Concurrency issues with ComponentProvider
On 2012-03-15 4:25, Thorsten Scherler wrote: I think the key is the map:match pattern=**.* which is a welcome door for concurrency issues. You need to request the page around 10 times and you see image glitches. I started a small testing project but will need to finish it later. BTW I created https://issues.apache.org/jira/browse/COCOON3-93 to keep track of the issue. By the way, do you use Reloading Classloader (RCL) in your application? I've had too many issues with it totally breaking Spring context... If you use it it would be good to confirm that the problem exists without it.
RE: [C3] Concurrency issues with ComponentProvider
Hi Igor, Interesting to find out how you disabled it. I am myself facing issues with C2.2 and RCL. This might have been caused by upgrading to Maven3 which again seems to have issues with the maven-jetty-plugin. So after some googling yesterday I switched to a newer jetty-maven-plugin (yes... it's renamed ;-)) but this gave me following stack trace: [INFO] Webapp source directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\rcl\webapp [INFO] Reload Mechanic: automatic [INFO] Classes = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\classes [INFO] Context path = / [INFO] Tmp directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\tmp [INFO] Web defaults = org/eclipse/jetty/webapp/webdefault.xml [INFO] Web overrides = none [INFO] web.xml file = file:/C:/development/workspaces/intellij11/CTPI-PX/spider2/shared/target/rcl/webapp/WEB-INF/web.xml [INFO] Webapp directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\rcl\webapp 2012-03-14 16:32:33.863:INFO:oejs.Server:jetty-8.1.0.RC5 2012-03-14 16:32:35.421:INFO:oejpw.PlusConfiguration:No Transaction manager found - if your webapp requires one, please configure one. org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderCreationException: Error while creating the URLClassLoader from context://WEB-INF/cocoon/rclwrapper.urlc l.conf at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderManager.createURLClassLoader(ReloadingClassloaderManager.java:101) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderManager.getClassLoader(ReloadingClassloaderManager.java:50) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingListener.invoke(ReloadingListener.java:148) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingListener.attributeAdded(ReloadingListener.java:283) at org.eclipse.jetty.server.handler.ContextHandler$Context.setAttribute(ContextHandler.java:2040) at org.apache.jasper.runtime.TldScanner.onStartup(TldScanner.java:241) at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:100) at org.eclipse.jetty.annotations.ServletContainerInitializerListener.contextInitialized(ServletContainerInitializerListener.java:99) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:764) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:406) Does anyone know if the latest jetty plugin supports a nice reload mechanism as I noticed this INFO message: [INFO] Reload Mechanic: automatic Robby -Original Message- From: Igor Malinin [mailto:igor...@gmail.com] Sent: Thursday, March 15, 2012 9:13 AM To: dev@cocoon.apache.org Subject: Re: [C3] Concurrency issues with ComponentProvider On 2012-03-15 4:25, Thorsten Scherler wrote: I think the key is the map:match pattern=**.* which is a welcome door for concurrency issues. You need to request the page around 10 times and you see image glitches. I started a small testing project but will need to finish it later. BTW I created https://issues.apache.org/jira/browse/COCOON3-93 to keep track of the issue. By the way, do you use Reloading Classloader (RCL) in your application? I've had too many issues with it totally breaking Spring context... If you use it it would be good to confirm that the problem exists without it.
Re: [C3] Concurrency issues with ComponentProvider
I just removed all references to RCL from from the project and then used JRebel instead of it. I had to change then one line in XsltTransformer to make XSLTs reloadable (it skipped checking originally if XSLT came from a JAR). AFAIK Jetty is able to re-deploy application in case it detects changes. I don't know exactly how it does the detection and how to configure it, most probably it is checking for web.xml timestamp. On 2012-03-15 10:41, Robby Pelssers wrote: Hi Igor, Interesting to find out how you disabled it. I am myself facing issues with C2.2 and RCL. This might have been caused by upgrading to Maven3 which again seems to have issues with the maven-jetty-plugin. So after some googling yesterday I switched to a newer jetty-maven-plugin (yes... it's renamed ;-)) but this gave me following stack trace: [INFO] Webapp source directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\rcl\webapp [INFO] Reload Mechanic: automatic [INFO] Classes = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\classes [INFO] Context path = / [INFO] Tmp directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\tmp [INFO] Web defaults = org/eclipse/jetty/webapp/webdefault.xml [INFO] Web overrides = none [INFO] web.xml file = file:/C:/development/workspaces/intellij11/CTPI-PX/spider2/shared/target/rcl/webapp/WEB-INF/web.xml [INFO] Webapp directory = C:\development\workspaces\intellij11\CTPI-PX\spider2\shared\target\rcl\webapp 2012-03-14 16:32:33.863:INFO:oejs.Server:jetty-8.1.0.RC5 2012-03-14 16:32:35.421:INFO:oejpw.PlusConfiguration:No Transaction manager found - if your webapp requires one, please configure one. org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderCreationException: Error while creating the URLClassLoader from context://WEB-INF/cocoon/rclwrapper.urlc l.conf at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderManager.createURLClassLoader(ReloadingClassloaderManager.java:101) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingClassloaderManager.getClassLoader(ReloadingClassloaderManager.java:50) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingListener.invoke(ReloadingListener.java:148) at org.apache.cocoon.tools.rcl.wrapper.servlet.ReloadingListener.attributeAdded(ReloadingListener.java:283) at org.eclipse.jetty.server.handler.ContextHandler$Context.setAttribute(ContextHandler.java:2040) at org.apache.jasper.runtime.TldScanner.onStartup(TldScanner.java:241) at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:100) at org.eclipse.jetty.annotations.ServletContainerInitializerListener.contextInitialized(ServletContainerInitializerListener.java:99) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:764) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:406) Does anyone know if the latest jetty plugin supports a nice reload mechanism as I noticed this INFO message: [INFO] Reload Mechanic: automatic Robby
RE: [C3] Concurrency issues with ComponentProvider
Could you elaborate a bit further how to remove all references to RCL? Did you check out the Cocoon sources and modify those? And where should I look for these references? Or are you referring to just commenting out all these instructions in the .rcl files? Because the latter sure does not fix this apparently. Robby -Original Message- From: Igor Malinin [mailto:igor...@gmail.com] Sent: Thursday, March 15, 2012 9:55 AM To: dev@cocoon.apache.org Subject: Re: [C3] Concurrency issues with ComponentProvider I just removed all references to RCL from from the project and then used JRebel instead of it. I had to change then one line in XsltTransformer to make XSLTs reloadable (it skipped checking originally if XSLT came from a JAR). AFAIK Jetty is able to re-deploy application in case it detects changes. I don't know exactly how it does the detection and how to configure it, most probably it is checking for web.xml timestamp.
Re: [C3] Concurrency issues with ComponentProvider
Hi Igor El 15 de marzo de 2012 09:12, Igor Malinin igor...@gmail.com escribió: On 2012-03-15 4:25, Thorsten Scherler wrote: I think the key is the map:match pattern=**.* which is a welcome door for concurrency issues. You need to request the page around 10 times and you see image glitches. I started a small testing project but will need to finish it later. BTW I created https://issues.apache.org/**jira/browse/COCOON3-93https://issues.apache.org/jira/browse/COCOON3-93to keep track of the issue. By the way, do you use Reloading Classloader (RCL) in your application? I've had too many issues with it totally breaking Spring context... If you use it it would be good to confirm that the problem exists without it. Yeah, I've experienced some issues with RCL before. It was our first option but we have the same problems deploying the war in Apache Tomcat for our test env so everything guide us to a Cocoon issue with concurrency.
Re: [C3] Concurrency issues with ComponentProvider
I don't remember exactly all the steps, but basically Cocoon RCL is the cocoon-maven-plugin, so you should remove this plugin from pom.xml then RCL will not be used. Commenting out instructions in .rcl files will not remove RCL itself from the build. No modification of Cocoon is required, Cocoon RCL just sits on top of 'normal' Cocoon. And you should not go to production with RCL anyway, so you should learn how to make builds that does not include RCL anyway... On 2012-03-15 10:58, Robby Pelssers wrote: Could you elaborate a bit further how to remove all references to RCL? Did you check out the Cocoon sources and modify those? And where should I look for these references? Or are you referring to just commenting out all these instructions in the .rcl files? Because the latter sure does not fix this apparently. Robby
RE: [C3] Concurrency issues with ComponentProvider
Hi Igor,
I commented out the cocoon-maven-plugin and now jetty starts fine. But now the
below configuration does not work anymore when executing mvn jetty:run
plugin
groupIdorg.mortbay.jetty/groupId
artifactIdmaven-jetty-plugin/artifactId !--
jetty-maven-plugin --
version6.1.26/version!-- 8.1.0.RC5 --
configuration
connectors
connector
implementation=org.mortbay.jetty.nio.SelectChannelConnector !--
org.eclipse.jetty.server.nio.SelectChannelConnector --
port/port
maxIdleTime3/maxIdleTime
/connector
/connectors
webAppSourceDirectory${project.build.directory}/rcl/webapp/webAppSourceDirectory
contextPath//contextPath
systemProperties
systemProperty
nameorg.apache.cocoon.mode/name
valuedev/value
/systemProperty
/systemProperties
/configuration
/plugin
It now can't find the webapp source directory anymore as I expect that the
cocoon-maven-plugin copied over files to the target/rcl folder.
So my next question is. How do you test (and start) an individual block?
[INFO] Final Memory: 26M/147M
[INFO]
[ERROR] Failed to execute goal org.mortbay.jetty:maven-jetty-plugin:6.1.26:run
(default-cli) on project shared: Webapp source directory
C:\development\workspaces\intell
ij11\CTPI-PX\spider2\shared\target\rcl\webapp does not exist - [Help 1]
-Original Message-
From: Igor Malinin [mailto:igor...@gmail.com]
Sent: Thursday, March 15, 2012 10:50 AM
To: dev@cocoon.apache.org
Subject: Re: [C3] Concurrency issues with ComponentProvider
I don't remember exactly all the steps, but basically Cocoon RCL is the
cocoon-maven-plugin, so you should remove this plugin from pom.xml then
RCL will not be used. Commenting out instructions in .rcl files will not
remove RCL itself from the build.
No modification of Cocoon is required, Cocoon RCL just sits on top of
'normal' Cocoon. And you should not go to production with RCL anyway, so
you should learn how to make builds that does not include RCL anyway...
On 2012-03-15 10:58, Robby Pelssers wrote:
Could you elaborate a bit further how to remove all references to RCL? Did
you check out the Cocoon sources and modify those? And where should I look
for these references? Or are you referring to just commenting out all these
instructions in the .rcl files? Because the latter sure does not fix this
apparently.
Robby
Re: [C3] Concurrency issues with ComponentProvider
I cannot look into this right now, but I'll sure do this evening. I had some frustrating problems with svn yesterday and after a full hour of not being able to do a simple svn up I gave up. Seems to work now, so I'll try again this evening...
Re: [C3] Concurrency issues with ComponentProvider
This should be removed without RCL:
webAppSourceDirectory${project.build.directory}/rcl/webapp/webAppSourceDirectory
I don't start individual blocks so I cannot help with this. But you can
always workaround it by creating web-app that references one block and
nothing else.
On 2012-03-15 12:13, Robby Pelssers wrote:
Hi Igor,
I commented out the cocoon-maven-plugin and now jetty starts fine. But now the
below configuration does not work anymore when executing mvn jetty:run
plugin
groupIdorg.mortbay.jetty/groupId
artifactIdmaven-jetty-plugin/artifactId !--
jetty-maven-plugin --
version6.1.26/version !-- 8.1.0.RC5 --
configuration
connectors
connector
implementation=org.mortbay.jetty.nio.SelectChannelConnector !--
org.eclipse.jetty.server.nio.SelectChannelConnector --
port/port
maxIdleTime3/maxIdleTime
/connector
/connectors
webAppSourceDirectory${project.build.directory}/rcl/webapp/webAppSourceDirectory
contextPath//contextPath
systemProperties
systemProperty
nameorg.apache.cocoon.mode/name
valuedev/value
/systemProperty
/systemProperties
/configuration
/plugin
It now can't find the webapp source directory anymore as I expect that the
cocoon-maven-plugin copied over files to the target/rcl folder.
So my next question is. How do you test (and start) an individual block?
Re: [C3] Concurrency issues with ComponentProvider
Hi Javier, that's the first time, I've heard of concurrency issues in C3. That's not to say there cannot be any, just that it's probably a rare case or you're stressing the system more than anyone else has yet. Some thoughts top off my head: - providers and factories should be singletons, as they should not have any state (ie. member variables) - the invocation must be a prototype, as it has a state - if synchronized reliably fixes the problem, then there is a problem with accessing the same, stateful resource from different threads I cannot look into this right now, but I'll sure do this evening. Are you using trunk or a certain build? I might be able to reproduce this on my own, but a reduced version (ie a controller and sitemap) for your setup would certainly help. Steven Am 14.03.2012 13:55, schrieb Javier Puerto: Hi Cocoon developers, we are working in a project based on C3 and we found a strange behaviour when loading the static resources. We are developing a Web 2.0 application and therefore we have a lot of resources (js, images and css). The weird thing is that sometimes the system is returning the wrong resources, returning a js instead of a css or switching images... For my tests I did a little modification in class FileReaderComponent to see the source of the file that is being served. See attached file FileReaderComponent.diff. This modification uses System.out to show the source of the file before and after read it. The component seems to be fine, also it's very simple so it seems to not be the cause. The patch allow me to discard the component as cause of problem, the source passed as argument when there's too much resources requests are wrong sometimes. I've review how the components are instantiated and I saw a possible design problem for ComponentProvider class. bean name=org.apache.cocoon.sitemap.ComponentProvider class=org.apache.cocoon.sitemap.SpringComponentProvider property name=actionFactory ref=org.apache.cocoon.sitemap.spring.ActionFactory / property name=languageInterpreterFactory ref=org.apache.cocoon.sitemap.expression.LanguageInterpreterFactory / property name=pipelineComponentFactory ref=org.apache.cocoon.sitemap.spring.PipelineComponentFactory / property name=pipelineFactory ref=org.apache.cocoon.sitemap.spring.PipelineFactory / /bean bean name=org.apache.cocoon.sitemap.Invocation class=org.apache.cocoon.sitemap.InvocationImpl scope=prototype property name=componentProvider ref=org.apache.cocoon.sitemap.ComponentProvider / /bean Above is the sitemap component configuration, the InvocationImpl is the component that will create the different components thought the ComponentProvider that will delegate on different factories. The problem I see is that while the Invocation is declared as prototype, the ComponentProvider is a shared instance so it could be that on high demand the methods overlaps between request. I've solved the problem declaring the methods as synchronized (att. SpringComponentProvider.diff). This will solve the problems with concurrency, but I wonder if it's the best solution. WDYT? It's very hard to reproduce the problem with tools like JMeter (better reloading directly in browser). Anyways it's happend sometimes with my current functional test plan but I can't reproduce it consistently. Also the JMeter test plan is designed for our current web project so I can't attach it. The functional tests is configure as: * 30 threads * No delay between request * Ramp up 1s * Infinite loop * Random controller with 6 static requests, each request with size assertion. Please, tell me if you need more detailed tests or it's enough. IMO anybody could see the problem if there's a lot of concurrent resources involved. Salu2
Re: [C3] Concurrency issues with ComponentProvider
Thanks Steven
El 14 de marzo de 2012 15:37, Steven Dolg steven.d...@indoqa.com escribió:
Hi Javier,
that's the first time, I've heard of concurrency issues in C3.
That's not to say there cannot be any, just that it's probably a rare case
or you're stressing the system more than anyone else has yet.
Strange, we experienced this kind of problems since the beginning. For your
information,
we need 34 request to retrieve all the resources for the homepage (devel
environment).
Some thoughts top off my head:
- providers and factories should be singletons, as they should not have
any state (ie. member variables)
- the invocation must be a prototype, as it has a state
- if synchronized reliably fixes the problem, then there is a problem
with accessing the same, stateful resource from different threads
Yeah, I didn't investigate further but maybe the problem is only for
certain factory.
I cannot look into this right now, but I'll sure do this evening.
Are you using trunk or a certain build?
We are using trunk.
I might be able to reproduce this on my own, but a reduced version (ie a
controller and sitemap) for your setup would certainly help.
I could provide an example block but I don't know if I will have time till
next weekend. Here is how we declared the static resources in sitemap:
map:pipeline id=static-resources type=noncaching
!-- map:parameter name=cache-expires value=access plus 1
weeks/ --
!-- images --
map:match pattern=images/**
map:match pattern=**.*
map:read src=resources/{map:1}.{map:2} /
/map:match
/map:match
!-- css and js --
map:match pattern=**.*
map:read src=resources/{map:1}.{map:2} /
/map:match
/map:pipeline
I can confirm that the HTML page is well-formed (JMeter tests again) and
the problem is in Cocoon side (see FileReader patch for my tests). So to
reproduce the problem, a page with 30 img/ tags loading different images
should work and refresh sometimes without cache (ctrl+f5 in FF).
Steven
Salu2
Am 14.03.2012 13:55, schrieb Javier Puerto:
Hi Cocoon developers,
we are working in a project based on C3 and we found a strange behaviour
when loading the static resources. We are developing a Web 2.0 application
and therefore we have a lot of resources (js, images and css). The weird
thing is that sometimes the system is returning the wrong resources,
returning a js instead of a css or switching images...
For my tests I did a little modification in class FileReaderComponent to
see the source of the file that is being served. See attached file
FileReaderComponent.diff.
This modification uses System.out to show the source of the file before
and after read it. The component seems to be fine, also it's very simple so
it seems to not be the cause. The patch allow me to discard the component
as cause of problem, the source passed as argument when there's too much
resources requests are wrong sometimes.
I've review how the components are instantiated and I saw a possible
design problem for ComponentProvider class.
bean name=org.apache.cocoon.**sitemap.ComponentProvider
class=org.apache.cocoon.**sitemap.**SpringComponentProvider
property name=actionFactory
ref=org.apache.cocoon.**sitemap.spring.ActionFactory
/
property name=**languageInterpreterFactory ref=org.apache.cocoon.**
sitemap.expression.**LanguageInterpreterFactory /
property name=**pipelineComponentFactory ref=org.apache.cocoon.**
sitemap.spring.**PipelineComponentFactory /
property name=pipelineFactory ref=org.apache.cocoon.**sitemap.spring.
**PipelineFactory /
/bean
bean name=org.apache.cocoon.**sitemap.Invocation
class=org.apache.cocoon.**sitemap.InvocationImpl scope=prototype
property name=componentProvider
ref=org.apache.cocoon.**sitemap.ComponentProvider
/
/bean
Above is the sitemap component configuration, the InvocationImpl is the
component that will create the different components thought the
ComponentProvider that will delegate on different factories. The problem I
see is that while the Invocation is declared as prototype, the
ComponentProvider is a shared instance so it could be that on high demand
the methods overlaps between request. I've solved the problem declaring the
methods as synchronized (att. SpringComponentProvider.diff).
This will solve the problems with concurrency, but I wonder if it's the
best solution. WDYT?
It's very hard to reproduce the problem with tools like JMeter (better
reloading directly in browser). Anyways it's happend sometimes with my
current functional test plan but I can't reproduce it consistently. Also
the JMeter test plan is designed for our current web project so I can't
attach it. The functional tests is configure as:
* 30 threads
* No delay between request
* Ramp up 1s
* Infinite loop
* Random controller with 6 static requests, each request with size
assertion.
Please, tell me if you need more
Re: [C3] Concurrency issues with ComponentProvider
On 03/14/2012 04:17 PM, Javier Puerto wrote:
Thanks Steven
El 14 de marzo de 2012 15:37, Steven Dolg steven.d...@indoqa.com
mailto:steven.d...@indoqa.com escribió:
Hi Javier,
that's the first time, I've heard of concurrency issues in C3.
That's not to say there cannot be any, just that it's probably a
rare case or you're stressing the system more than anyone else has
yet.
Strange, we experienced this kind of problems since the beginning. For
your information,
we need 34 request to retrieve all the resources for the homepage
(devel environment).
I can confirm Javier's observations, we are colleagues and on the same
project. Further we experience some random exceptions in our c3 REST
services, like said random. Javier has a nose to track down concurrency
issues (see latest c2.2 patch) and uses the force of jmeter. ;)
...but even without jmeter we experience this strange behaviour.
Some thoughts top off my head:
- providers and factories should be singletons, as they should not
have any state (ie. member variables)
- the invocation must be a prototype, as it has a state
- if synchronized reliably fixes the problem, then there is a
problem with accessing the same, stateful resource from different
threads
Yeah, I didn't investigate further but maybe the problem is only for
certain factory.
Hmm, when Javier showed me the patch before submitting I raised my
eyebrows of the sync use in the methods, but I can confirm applying the
patch has fixed the weird image switching behaviour.
so I agree with Steve saying if synchronized reliably fixes the
problem, then there is a problem with accessing the same, stateful
resource from different threads but actually it is the access of the
underlying spring appcontext since nearly all factories are using it to
resolve the components. Meaning the access of the spring bean factory
seems not be in general synchronized.
I cannot look into this right now, but I'll sure do this evening.
Are you using trunk or a certain build?
We are using trunk.
It would be really nice to get some feedback on this since see above we
experience some general glitches in our app which could be provoked by
concurrency issues
I might be able to reproduce this on my own, but a reduced version
(ie a controller and sitemap) for your setup would certainly help.
I could provide an example block but I don't know if I will have time
till next weekend. Here is how we declared the static resources in
sitemap:
map:pipeline id=static-resources type=noncaching
!-- map:parameter name=cache-expires value=access plus 1 weeks/ --
!-- images --
map:match pattern=images/**
map:match pattern=**.*
map:read src=resources/{map:1}.{map:2} /
/map:match
/map:match
!-- css and js --
map:match pattern=**.*
map:read src=resources/{map:1}.{map:2} /
/map:match
/map:pipeline
I can confirm that the HTML page is well-formed (JMeter tests again)
and the problem is in Cocoon side (see FileReader patch for my tests).
So to reproduce the problem, a page with 30 img/ tags loading
different images should work and refresh sometimes without cache
(ctrl+f5 in FF).
I think the key is the map:match pattern=**.* which is a welcome
door for concurrency issues. You need to request the page around 10
times and you see image glitches.
I started a small testing project but will need to finish it later. BTW
I created https://issues.apache.org/jira/browse/COCOON3-93 to keep track
of the issue.
salu2
--
Thorsten Scherlerscherler.at.gmail.com
codeBusters S.L. - web based systems
consulting, training and solutions
http://www.codebusters.es/
