Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Joerg Heinicke wrote: On 11.03.2008 08:54, Carsten Ziegeler wrote: We could argue about another default value than -1 though. Something like 1024^2. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. Hmm, ok, we could change this in the main sitemap as a default configuration while leaving it in the java code untouched. However, I still think that this is not needed, if people want to stream huge responses, they should think about what they are doing and configure everything accordingly. I totally agree that we lack documentation here. I really don't agree with this reasoning but I don't want to stress it too much to not get on your nerves, so it's my last try :-) Hehe :) Ok, and I try hard that this is my last response :) 1. This IS already documented in the main sitemap and I think though hardly anybody is aware of it: Nobody had reacted on Felix' issue entry for 3 weeks. I can also see why: our main sitemap is the first thing I would get rid of when creating my own Cocoon application ;-) 2. Actually I don't care about the huge resources that much. I rather have big resources in mind - and concurrent users. What's worse as a default behavior for an incautious user: a screwed error page or a server brought down with an OOME? I also wanted to put the default as big as 1 MB so that it hardly affects anybody. I really don't see an additional value of endless buffering over such a big buffer. But I can prevent a fatal user error. And for everything that's beyond the default buffer size, here I completely agree with you, the user should think about potential issues in terms of buffering or caching. 3. Another, I admit minor point after so long time might be the change in behavior compared to former Cocoon versions. According to Robert [1] in Cocoon 2.0.4 it used to work out of the box. Yes, we discussed this for 2.1.0 and decided that it's better to have a correct error response per default. Ok, I agree that the difference is really very subtle, so lets change it; I guess one mb should be enough. I guess you want to change it for 2.1.x as well, right? Carsten -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 11.03.2008 08:54, Carsten Ziegeler wrote: We could argue about another default value than -1 though. Something like 1024^2. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. Hmm, ok, we could change this in the main sitemap as a default configuration while leaving it in the java code untouched. However, I still think that this is not needed, if people want to stream huge responses, they should think about what they are doing and configure everything accordingly. I totally agree that we lack documentation here. I really don't agree with this reasoning but I don't want to stress it too much to not get on your nerves, so it's my last try :-) 1. This IS already documented in the main sitemap and I think though hardly anybody is aware of it: Nobody had reacted on Felix' issue entry for 3 weeks. I can also see why: our main sitemap is the first thing I would get rid of when creating my own Cocoon application ;-) 2. Actually I don't care about the huge resources that much. I rather have big resources in mind - and concurrent users. What's worse as a default behavior for an incautious user: a screwed error page or a server brought down with an OOME? I also wanted to put the default as big as 1 MB so that it hardly affects anybody. I really don't see an additional value of endless buffering over such a big buffer. But I can prevent a fatal user error. And for everything that's beyond the default buffer size, here I completely agree with you, the user should think about potential issues in terms of buffering or caching. 3. Another, I admit minor point after so long time might be the change in behavior compared to former Cocoon versions. According to Robert [1] in Cocoon 2.0.4 it used to work out of the box. Joerg [1] http://marc.info/?l=xml-cocoon-devm=120473398413381w=4
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Joerg Heinicke wrote: On 05.03.2008 23:06, Joerg Heinicke wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. I think also if you're using a reader in your pipeline it is more likely that you don't want to let the pipeline buffer your output. Carsten Joerg [1] https://issues.apache.org/jira/browse/COCOON-2168 -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Felix Knecht schrieb: Carsten Ziegeler schrieb: Joerg Heinicke wrote: On 05.03.2008 23:06, Joerg Heinicke wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. I think also if you're using a reader in your pipeline it is more likely that you don't want to let the pipeline buffer your output. IIUC this should work (no caching pipeline, set buffer-size to 8192): map:pipeline id=test-nocache type=noncaching map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline but it doesn't java.lang.OutOfMemoryError: Java heap space You need to turn off the buffering of the pipeline as well. I don't have the parameter name at hand, I assume it's buffer-size as well but could be different: map:pipeline id=test-nocache type=noncaching map:parameter name=buffer-size value=0 / map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline This is usually the way I would define reader pipelines. If the above still produces an OOMError than we have a bug :) Carsten -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Felix Knecht wrote: You need to turn off the buffering of the pipeline as well. I don't have the parameter name at hand, I assume it's buffer-size as well but could be different: map:pipeline id=test-nocache type=noncaching map:parameter name=buffer-size value=0 / map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline This is usually the way I would define reader pipelines. If the above still produces an OOMError than we have a bug :) Thanks Carsten, it works. Parameter is outputBufferSize Ah, yes :) Great! I may ask at this point if the general configuration of noncaching pipelines is correct (cocoon-core/cocoon-core/src/main/resources/META-INF/cocoon/avalon/cocoon-core-sitemapcomponents.xconf): map:pipe name=noncaching src=org.apache.cocoon.components.pipeline.impl.NonCachingProcessingPipeline !-- parameter name=outputBufferSize value=8192/ -- /map:pipe == As no parameter is specified '-1' is used what in fact leads to the same configuration as for caching pipelines?! Yes, this is correct :) The output buffer only specifies the size of the buffer for writing the response. This is not directly related to caching. You might increase performance by buffering. The buffer is in both case unlimited because of allowing proper error handling. If we don't have it yet, we should add these things to a tuning cocoon page. I would turn off infinite buffering in production in all case and set a fixed buffer size (like 8192). For reader pipelines I would turn off buffering completly. Carsten -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
You need to turn off the buffering of the pipeline as well. I don't have the parameter name at hand, I assume it's buffer-size as well but could be different: map:pipeline id=test-nocache type=noncaching map:parameter name=buffer-size value=0 / map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline This is usually the way I would define reader pipelines. If the above still produces an OOMError than we have a bug :) Thanks Carsten, it works. Parameter is outputBufferSize map:pipeline id=test-nocache type=noncaching map:parameter name=outputBufferSize value=0 / map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline I may ask at this point if the general configuration of noncaching pipelines is correct (cocoon-core/cocoon-core/src/main/resources/META-INF/cocoon/avalon/cocoon-core-sitemapcomponents.xconf): map:pipe name=noncaching src=org.apache.cocoon.components.pipeline.impl.NonCachingProcessingPipeline !-- parameter name=outputBufferSize value=8192/ -- /map:pipe == As no parameter is specified '-1' is used what in fact leads to the same configuration as for caching pipelines?! Felix
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Carsten Ziegeler schrieb: Joerg Heinicke wrote: On 05.03.2008 23:06, Joerg Heinicke wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. I think also if you're using a reader in your pipeline it is more likely that you don't want to let the pipeline buffer your output. IIUC this should work (no caching pipeline, set buffer-size to 8192): map:pipeline id=test-nocache type=noncaching map:match pattern=nocache map:read src=/home/felix/tmp/livecd-i686-installer-2007.0.iso map:parameter name=buffer-size value=8192 / /map:read /map:match /map:pipeline but it doesn't java.lang.OutOfMemoryError: Java heap space at org.apache.cocoon.util.BufferedOutputStream.incBuffer(BufferedOutputStream.java:148) at org.apache.cocoon.util.BufferedOutputStream.write(BufferedOutputStream.java:96) at org.apache.cocoon.reading.ResourceReader.processStream(ResourceReader.java:355) at org.apache.cocoon.reading.ResourceReader.generate(ResourceReader.java:386) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.processReader(AbstractProcessingPipeline.java:656) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:431) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:72) at $Proxy8.process(Unknown Source) at org.apache.cocoon.components.treeprocessor.sitemap.ReadNode.invoke(ReadNode.java:94) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55) at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78) at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:144) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78) at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81) at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239) at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:171) at org.apache.cocoon.components.treeprocessor.TreeProcessor.process(TreeProcessor.java:247) at org.apache.cocoon.servlet.RequestProcessor.process(RequestProcessor.java:351) at org.apache.cocoon.servlet.RequestProcessor.service(RequestProcessor.java:169) at org.apache.cocoon.sitemap.SitemapServlet.service(SitemapServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:501) at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:473) at org.apache.cocoon.servletservice.spring.ServletFactoryBean$ServiceInterceptor.invoke(ServletFactoryBean.java:230) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at $Proxy5.service(Unknown Source) Felix Carsten Joerg [1] https://issues.apache.org/jira/browse/COCOON-2168
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 11.03.2008 04:48, Carsten Ziegeler wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). But I fear hardly anybody is aware or even uses the feature. The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. It's not really development, is it? I mean even if you can not reset output buffer completely, you will still get the error markup appended and I would not care for development about how this looks :) Being aware of the potential change in behavior I also chose a quite huge buffer of 1 MB so that hardly anybody should be affected. We can also discuss about making it even bigger like 10 MB. But I consider a buffer that's flushed too early once in a while better than an OOME in the default setup. And people can still change it to -1 and endless buffering if they really need. But at least they are aware of the effects then. Joerg
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Joerg Heinicke wrote: On 11.03.2008 04:48, Carsten Ziegeler wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Hmm, not sure if we should change the default value. The idea of this default was to be sure that error handling works out of the box. If you have special cases like mentioned in the bug, it makes imho more sense to fine tune these special cases (for instance by not buffering). But I fear hardly anybody is aware or even uses the feature. Yes, that's possible. The output buffer value is one of the settings which is optimized for development and it should be tweaked for production usage. It's not really development, is it? I mean even if you can not reset output buffer completely, you will still get the error markup appended and I would not care for development about how this looks :) Hmm, I would never rely on the default error handling for these cases in production environments. If something is already written to the output stream, it's too late anyway. But I see your point. Being aware of the potential change in behavior I also chose a quite huge buffer of 1 MB so that hardly anybody should be affected. We can also discuss about making it even bigger like 10 MB. But I consider a buffer that's flushed too early once in a while better than an OOME in the default setup. And people can still change it to -1 and endless buffering if they really need. But at least they are aware of the effects then. Hmm, ok, we could change this in the main sitemap as a default configuration while leaving it in the java code untouched. However, I still think that this is not needed, if people want to stream huge responses, they should think about what they are doing and configure everything accordingly. I totally agree that we lack documentation here. Carsten -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Carsten Ziegeler escribió: Hmm, ok, we could change this in the main sitemap as a default configuration while leaving it in the java code untouched. However, I still think that this is not needed, if people want to stream huge responses, they should think about what they are doing and configure everything accordingly. I totally agree that we lack documentation here. +1 Best Regards, Antonio Gallardo.
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Felix Knecht wrote: Joerg Heinicke schrieb: On 05.03.2008 23:06, Joerg Heinicke wrote: From what I see from the code (AbstractProcessingPipeline) it is possible to configure and setup/parameterize a pipeline with outputBufferSize. This means on both map:pipe and map:pipeline it should be possible to set an actual buffer size. Only if none is set (and it defaults to -1) the non-flushing BufferedOutputStream is used. I found a thread on the users list claiming outputBufferSize does not work [1] at the moment (or since some time). Carsten wrote he sees a potential problem but does not outline them. ATM we are always talking about Buffered... I'm believe this has to do with caching the pipelines, right? So I wonder really why my the issue also occurs in noncaching pipelines? Sorry for the late response. I thought that the output stream handling implementation in AbstractEnvironment is wrong. But looking at the code again I'm not sure anymore :( For whatever reason I had the thought the multiple calls to getOutputStream(int) - perhaps with different buffer sizes - could reveal a wrong output stream to be used. Don't know if this is the actual problem, but I think we should throw an exception if getOutputStream is called twice. It should only be called once during a request. Carsten -- Carsten Ziegeler [EMAIL PROTECTED]
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 05.03.2008 23:06, Joerg Heinicke wrote: From what I see from the code (AbstractProcessingPipeline) it is possible to configure and setup/parameterize a pipeline with outputBufferSize. This means on both map:pipe and map:pipeline it should be possible to set an actual buffer size. Only if none is set (and it defaults to -1) the non-flushing BufferedOutputStream is used. I found a thread on the users list claiming outputBufferSize does not work [1] at the moment (or since some time). Carsten wrote he sees a potential problem but does not outline them. Felix, have you tried outputBufferSize? Carsten, can you tell us what might be wrong? Joerg [1] http://marc.info/?t=120111212300011r=1w=4
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Joerg Heinicke schrieb: On 05.03.2008 23:06, Joerg Heinicke wrote: From what I see from the code (AbstractProcessingPipeline) it is possible to configure and setup/parameterize a pipeline with outputBufferSize. This means on both map:pipe and map:pipeline it should be possible to set an actual buffer size. Only if none is set (and it defaults to -1) the non-flushing BufferedOutputStream is used. I found a thread on the users list claiming outputBufferSize does not work [1] at the moment (or since some time). Carsten wrote he sees a potential problem but does not outline them. Felix, have you tried outputBufferSize? Not yet. I didn't even knew how to configure it :-( Felix Carsten, can you tell us what might be wrong? Joerg [1] http://marc.info/?t=120111212300011r=1w=4
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Joerg Heinicke schrieb: On 05.03.2008 23:06, Joerg Heinicke wrote: From what I see from the code (AbstractProcessingPipeline) it is possible to configure and setup/parameterize a pipeline with outputBufferSize. This means on both map:pipe and map:pipeline it should be possible to set an actual buffer size. Only if none is set (and it defaults to -1) the non-flushing BufferedOutputStream is used. I found a thread on the users list claiming outputBufferSize does not work [1] at the moment (or since some time). Carsten wrote he sees a potential problem but does not outline them. ATM we are always talking about Buffered... I'm believe this has to do with caching the pipelines, right? So I wonder really why my the issue also occurs in noncaching pipelines? Felix
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 08.03.2008 15:34, Felix Knecht wrote: ATM we are always talking about Buffered... I'm believe this has to do with caching the pipelines, right? So I wonder really why my the issue also occurs in noncaching pipelines? No, it has to do with the capability of resetting the OutputStream to send an error response in case of an error. If your buffer is too small, a first part of the actual content might already have been flushed when the error occurs and the error content is appended. Looks weird at the client's browser ;) Joerg
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 08.03.2008 15:28, Felix Knecht wrote: Felix, have you tried outputBufferSize? Not yet. I didn't even knew how to configure it :-( Both map:pipe and map:pipeline should work as mentioned here: http://marc.info/?l=xml-cocoon-devm=120477640924395w=4 Joerg
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Antonio Gallardo pisze: 1MB Makes sense. +1 -- Best regards, Grzegorz Kossakowski
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 05.03.2008 23:06, Joerg Heinicke wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Joerg [1] https://issues.apache.org/jira/browse/COCOON-2168
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
1MB Makes sense. Best Regards, Antonio Gallardo. Joerg Heinicke escribió: On 05.03.2008 23:06, Joerg Heinicke wrote: We could argue about another default value than -1 though. Something like 1024^2. What do others think? Shall we change the default value from buffer everything (which lead to the OutOfMemoryError [1]) to something more secure in the sense of avoiding a potential source of error? Besides mentioning it on the changes page we have to set it to a value that's unlikely to be hit with a normal web application to change user application's behavior only in extreme cases. That's why I suggested 1MB. Joerg [1] https://issues.apache.org/jira/browse/COCOON-2168
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Robert To get an answer to such questions the cocoon developer list would be the right place (i'll do it for you this time as cc)! That's what the lists are for! See also [1] for cocoons mailing lists [1] http://cocoon.apache.org/1275_1_1.html We are encountering this same problem in Cocoon 2.1.11. What is the status of this issue? Is there a fix for it? It doesn't seem to exist in Cocoon 2.0.4. Obviously net yet, otherwise you could see it in the issue tracker. Maybe BufferedOutputStream was introduced after cocoon 2.0.4 so that version doesn't suffers this issue. Ciao Felix https://issues.apache.org/jira/browse/COCOON-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12568953#action_12568953 Sincerely, Robert La Ferla OMS
Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
On 05.03.2008 11:19, Felix Knecht wrote:
We are encountering this same problem in Cocoon 2.1.11. What is the
status of this issue? Is there a fix for it? It doesn't seem to
exist in Cocoon 2.0.4.
Obviously net yet, otherwise you could see it in the issue tracker.
Maybe BufferedOutputStream was introduced after cocoon 2.0.4 so that
version doesn't suffers this issue.
I've looked into this issue and I'm against Felix' fix. It stands
completely against the idea of buffering the whole pipeline content
which is done for error handlers. It should be possible for them to
reset the OutputStream. Or asked in other words? Why using the
non-flushing BufferedOutputStream at all?
Now how to handle the issue with huge resources? Should be quite easy.
From what I see from the code (AbstractProcessingPipeline) it is
possible to configure and setup/parameterize a pipeline with
outputBufferSize. This means on both map:pipe and map:pipeline it
should be possible to set an actual buffer size. Only if none is set
(and it defaults to -1) the non-flushing BufferedOutputStream is used.
Root sitemap has this example (in map:components/map:pipes section):
map:pipe name=noncaching
src=org.apache.cocoon.components.pipeline.impl.NonCachingProcessingPipeline
pool-max=${noncaching-pipeline.pool-max}
!-- parameter name=outputBufferSize value=8192/ --
/map:pipe
The following should also work (in map:pipelines section):
map:pipeline
map:parameter name=outputBufferSize value=8192/
...
/map:pipeline
We could argue about another default value than -1 though. Something
like 1024^2.
WDYT?
Joerg
