Hi, I worked before on doing multithreading in the ContentAggregator, but whith out succes.
I had a modified ContentAggregator class with Part extending thread, getting all output buffered, and a second loop in the aggregators generate-method to join and stream the buffers. This solution did work in c-2.0.4, but not in c2.1m3. It sometimes throws exceptions during the release of components. It the meantime there was some interest from Tony Culshaw about this issue. I sent him my aggregator, but he came up with his own way of doing it. It was mainly the same as my SAX-buffer-solution, but storing the parts as DOM and then stream them. Carsten Ziegeler gave me the advise to have a look at the CIncludeTransformer, where parallel retrieval of source should work. I tried it out. I wrote a aggregator that uses exactly the same IncludeCacheManager/Session mechanism as CIncludeTransformer. When I switch the parameter "parallel" for IncludeCacheManagerSession to "true", I run into the same problem as before. For parallel=false ist works. So it seems to me that the problem is somewhere deeper in cocoon or avalon code. He wrote: > multi-threading in Cocoon is unfortunately a little bit difficult. > Cocoon uses some ThreadLocal variables that have to set and used > properly in any subtasks I have no idea witch variables that could be. I thing some method for the release has to be decleare as synchronized. Has anybody any idea? As I understand the code, it does (by useing the cache manager/session classes) nothing more with the variables then when I have two loops, the first that starts the source threads, using SourceUtil.toSAX to fetch the content, the second loop joins the threads and streams them. What is the cache manager good for regarding multithreating? I think everything, that worked in cocoon 2.0.4, should work in 2.1. Especialy a that importen feature as multithreating. I would apriciate any help, I'm totaly confused at the moment. Regards Christoph Gaffga [EMAIL PROTECTED] P.S.: If anybody want to have a look on the aggregator code, I can send you the java file (dont want attachement on the mailinglist). P.P.S.: The Exception thrown (sometimes, depends on the pool I think) java.lang.IndexOutOfBoundsException: Index: 23, Size: 23 at java.util.ArrayList.RangeCheck(ArrayList.java:507) at java.util.ArrayList.remove(ArrayList.java:392) at org.apache.cocoon.components.EnvironmentDescription.removeFromAutoRelease(Co coonComponentManager.java:620) at org.apache.cocoon.components.CocoonComponentManager.removeFromAutomaticRelea se(CocoonComponentManager.java:420) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.release(Abs tractProcessingPipeline.java:203) at org.apache.cocoon.components.source.impl.SitemapSource.reset(SitemapSource.j ava:454) at org.apache.cocoon.components.source.impl.SitemapSource.recycle(SitemapSource .java:474) at org.apache.cocoon.components.source.impl.SitemapSourceFactory.release(Sitema pSourceFactory.java:113) at org.apache.excalibur.source.impl.SourceResolverImpl.release(SourceResolverIm pl.java:308) at org.apache.cocoon.components.CocoonComponentManager.release(CocoonComponentM anager.java:467) at org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen t.java:521) at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:317) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs tractProcessingPipeline.java:637) at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline .recycle(AbstractCachingProcessingPipeline.java:959) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.EnvironmentDescription.removeFromAutoRelease(Co coonComponentManager.java:615) at org.apache.cocoon.components.CocoonComponentManager.removeFromAutomaticRelea se(CocoonComponentManager.java:420) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.release(Abs tractProcessingPipeline.java:203) at org.apache.cocoon.components.source.impl.SitemapSource.reset(SitemapSource.j ava:454) at org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.j ava:446) at org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:159) at org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:126) at org.apache.cocoon.transformation.helpers.LoaderThread.run(DefaultIncludeCach eManager.java:467) at java.lang.Thread.run(Thread.java:534) java.lang.NullPointerException at org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen t.java:521) at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:317) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs tractProcessingPipeline.java:637) at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline .recycle(AbstractCachingProcessingPipeline.java:959) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.EnvironmentDescription.release(CocoonComponentM anager.java:547) at org.apache.cocoon.components.CocoonComponentManager.endProcessing(CocoonComp onentManager.java:199) at org.apache.cocoon.Cocoon.process(Cocoon.java:641) at org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.java:1083) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher. java:684) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatch er.java:432) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher .java:356) at com.triplemind.asp.UrlMappingServlet.service(UrlMappingServlet.java:314) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:247) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:193) at com.triplemind.asp.server.AuthFilter.doFilter(AuthFilter.java:100) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:213) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:193) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:260) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:191) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180 ) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve. java:170) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:641) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172 ) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:641) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :174) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne ction(Http11Protocol.java:386) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav a:530) at java.lang.Thread.run(Thread.java:534) java.lang.NullPointerException at org.apache.cocoon.environment.AbstractEnvironment.release(AbstractEnvironmen t.java:521) at org.apache.cocoon.generation.FileGenerator.recycle(FileGenerator.java:90) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:317) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.recycle(Abs tractProcessingPipeline.java:637) at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline .recycle(AbstractCachingProcessingPipeline.java:959) at org.apache.avalon.excalibur.pool.ResourceLimitingPool.put(ResourceLimitingPo ol.java:438) at org.apache.avalon.excalibur.component.PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:245) at org.apache.avalon.excalibur.component.ComponentHandler.put(ComponentHandler. java:452) at org.apache.avalon.excalibur.component.ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:336) at org.apache.cocoon.components.ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:320) at org.apache.cocoon.components.EnvironmentDescription.release(CocoonComponentM anager.java:547) at org.apache.cocoon.components.CocoonComponentManager.endProcessing(CocoonComp onentManager.java:199) at org.apache.cocoon.Cocoon.process(Cocoon.java:641) at org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.java:1083) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher. java:684) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatch er.java:432) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher .java:356) at com.triplemind.asp.UrlMappingServlet.service(UrlMappingServlet.java:314) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:247) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:193) at com.triplemind.asp.server.AuthFilter.doFilter(AuthFilter.java:100) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:213) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:193) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:260) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:191) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180 ) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve. java:170) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:641) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172 ) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:641) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :174) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok eNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne ction(Http11Protocol.java:386) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav a:530) at java.lang.Thread.run(Thread.java:534)
