Re: HttpProducer endpoint with unlimited/limited redelivery - memory leak
Hi
Thanks for reporting. Willem logged and fixed this
https://issues.apache.org/jira/browse/CAMEL-8134
On Mon, Dec 8, 2014 at 11:53 AM, Radek Kraus radek_kr...@centrum.cz wrote:
Hello,
I have a suspicion on (or better I faced to) memory leak in the following
situation. I use HttpProducer endpoint with unlimited redelivery
(configuration problem), but the problem is present for limited redelivery
too. Here is simplified route, which I used for problem simulation (finally
I got OOM in this simulation):
return new RouteBuilder() {
public void configure() {
errorHandler(
defaultErrorHandler().maximumRedeliveries(-1).redeliveryDelay(5).maximumRedeliveryDelay(5)
);
from(direct:start).setHeader(Exchange.HTTP_METHOD,
POST).to(http://localhost:8899/test;);
}
};
The problem occurs, when HTTP server returns 404 as response code (for
example when the target web application is undeployed from web application
container). In this situation the next/new synchronization callback is
registered in each HttpProducer endpoint processing, which caused that we
have a lot of close synchronization actions registered on UnitOfWork in
some time (OOM problem in extreme situation) - see
org.apache.camel.converter.stream.CachedOutputStream(Exchange exchange,
final boolean closedOnCompletion, which is invoked from HttpProducer, method
doExtractResponseBodyAsStream(...))
Maybe the problem relates to CAMEL-7055, where the synchronization callback
is registered even if closedOnCompletion parameter is false.
I see, that stream close synchronization is necessary, mainly for
situation, where we got the valid body response, but maybe for
populateHttpOperationFailedException isn't necessary to register the
synchronization action at all, because the response body is only converted
into java.lang.String.
I would like to ask, if my solution with using redelivery, is based on wrong
idea or I discovered a bug, which should be reported to JIRA.
Thank you for any feedback.
Radek Kraus.
--
View this message in context:
http://camel.465427.n5.nabble.com/HttpProducer-endpoint-with-unlimited-limited-redelivery-memory-leak-tp5760285.html
Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-
Red Hat, Inc.
Email: cib...@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: HttpProducer endpoint with unlimited/limited redelivery - memory leak
Hi Radek,
I revisit the code twice and found out we should not add the synchronisation if
the closedOnCompletion option is false. As it’s the user responsibility to
clean up the stream once it close the stream. In you case the exchange
synchronisation is keeping adding even the under layer input stream is consumed.
I created a JIRA[1] for it and committed a quick patch for it.
[1]https://issues.apache.org/jira/browse/CAMEL-8134
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 8, 2014 at 6:55:17 PM, Radek Kraus (radek_kr...@centrum.cz) wrote:
Hello,
I have a suspicion on (or better I faced to) memory leak in the following
situation. I use HttpProducer endpoint with unlimited redelivery
(configuration problem), but the problem is present for limited redelivery
too. Here is simplified route, which I used for problem simulation (finally
I got OOM in this simulation):
return new RouteBuilder() {
public void configure() {
errorHandler(
defaultErrorHandler().maximumRedeliveries(-1).redeliveryDelay(5).maximumRedeliveryDelay(5)
);
from(direct:start).setHeader(Exchange.HTTP_METHOD,
POST).to(http://localhost:8899/test;);
}
};
The problem occurs, when HTTP server returns 404 as response code (for
example when the target web application is undeployed from web application
container). In this situation the next/new synchronization callback is
registered in each HttpProducer endpoint processing, which caused that we
have a lot of close synchronization actions registered on UnitOfWork in
some time (OOM problem in extreme situation) - see
org.apache.camel.converter.stream.CachedOutputStream(Exchange exchange,
final boolean closedOnCompletion, which is invoked from HttpProducer, method
doExtractResponseBodyAsStream(...))
Maybe the problem relates to CAMEL-7055, where the synchronization callback
is registered even if closedOnCompletion parameter is false.
I see, that stream close synchronization is necessary, mainly for
situation, where we got the valid body response, but maybe for
populateHttpOperationFailedException isn't necessary to register the
synchronization action at all, because the response body is only converted
into java.lang.String.
I would like to ask, if my solution with using redelivery, is based on wrong
idea or I discovered a bug, which should be reported to JIRA.
Thank you for any feedback.
Radek Kraus.
--
View this message in context:
http://camel.465427.n5.nabble.com/HttpProducer-endpoint-with-unlimited-limited-redelivery-memory-leak-tp5760285.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: HttpProducer endpoint with unlimited/limited redelivery - memory leak
Hi Willem and Claus, many thank you for fix and JIRA issue too. I felt that one possibility, was to don't register synchronization at all (like was in version 2.12.2), but I wasn't sure with this option (mainly in context of CAMEL-7055). -- View this message in context: http://camel.465427.n5.nabble.com/HttpProducer-endpoint-with-unlimited-limited-redelivery-memory-leak-tp5760285p5760424.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: HttpProducer endpoint with unlimited/limited redelivery - memory leak
Hi Radek, It’s more complex than you thought. We need to clean up the cached the file in some place. I just added some unit tests to make sure we don’t introduce regress issue again. You can check out the patch if you are interesting about the whole context :) -- Willem Jiang Red Hat, Inc. Web: http://www.redhat.com Blog: http://willemjiang.blogspot.com (English) http://jnn.iteye.com (Chinese) Twitter: willemjiang Weibo: 姜宁willem On December 10, 2014 at 3:29:57 AM, Radek Kraus (radek_kr...@centrum.cz) wrote: Hi Willem and Claus, many thank you for fix and JIRA issue too. I felt that one possibility, was to don't register synchronization at all (like was in version 2.12.2), but I wasn't sure with this option (mainly in context of CAMEL-7055). -- View this message in context: http://camel.465427.n5.nabble.com/HttpProducer-endpoint-with-unlimited-limited-redelivery-memory-leak-tp5760285p5760424.html Sent from the Camel - Users mailing list archive at Nabble.com.
HttpProducer endpoint with unlimited/limited redelivery - memory leak
Hello,
I have a suspicion on (or better I faced to) memory leak in the following
situation. I use HttpProducer endpoint with unlimited redelivery
(configuration problem), but the problem is present for limited redelivery
too. Here is simplified route, which I used for problem simulation (finally
I got OOM in this simulation):
return new RouteBuilder() {
public void configure() {
errorHandler(
defaultErrorHandler().maximumRedeliveries(-1).redeliveryDelay(5).maximumRedeliveryDelay(5)
);
from(direct:start).setHeader(Exchange.HTTP_METHOD,
POST).to(http://localhost:8899/test;);
}
};
The problem occurs, when HTTP server returns 404 as response code (for
example when the target web application is undeployed from web application
container). In this situation the next/new synchronization callback is
registered in each HttpProducer endpoint processing, which caused that we
have a lot of close synchronization actions registered on UnitOfWork in
some time (OOM problem in extreme situation) - see
org.apache.camel.converter.stream.CachedOutputStream(Exchange exchange,
final boolean closedOnCompletion, which is invoked from HttpProducer, method
doExtractResponseBodyAsStream(...))
Maybe the problem relates to CAMEL-7055, where the synchronization callback
is registered even if closedOnCompletion parameter is false.
I see, that stream close synchronization is necessary, mainly for
situation, where we got the valid body response, but maybe for
populateHttpOperationFailedException isn't necessary to register the
synchronization action at all, because the response body is only converted
into java.lang.String.
I would like to ask, if my solution with using redelivery, is based on wrong
idea or I discovered a bug, which should be reported to JIRA.
Thank you for any feedback.
Radek Kraus.
--
View this message in context:
http://camel.465427.n5.nabble.com/HttpProducer-endpoint-with-unlimited-limited-redelivery-memory-leak-tp5760285.html
Sent from the Camel - Users mailing list archive at Nabble.com.
