Re: StreamCaching in Camel 1.6
Hi
Just wanted to note that the bug is now fixed in the next release of
Camel - 1.6.1 and 2.0
Thanks for reporting.
On Tue, Mar 17, 2009 at 9:28 AM, Claus Ibsen claus.ib...@gmail.com wrote:
Hi
A quick test shows:
JMX disabled:
EventDrivenConsumerRoute[Endpoint[direct:start] -
UnitOfWork(StreamCachingInterceptor(choice{when body contains Hello:
Pipeline[DeadLetterChannel[sendTo(Endpoint[mock:hello]),
log:org.apache.camel.DeadLetterChannel?level=error],
DeadLetterChannel[StreamCachingInterceptor(choice{when body contains
Bye: DeadLetterChannel[StreamCachingInterceptor(sendTo(Endpoint[mock:bye])),
log:org.apache.camel.DeadLetterChannel?level=error]}),
log:org.apache.camel.DeadLetterChannel?level=error]]}))]
JMX enabled:
EventDrivenConsumerRoute[Endpoint[direct:start] -
UnitOfWork(Instrumentation(Instrumentation(choice{when body contains
Hello: Pipeline[sendTo(Endpoint[mock:hello]), choice{when body
contains Bye: sendTo(Endpoint[mock:bye])}]})))]
So yeah there is a bug in Camel that is affected by the JMX.
I will log a JIRA for this.
https://issues.apache.org/activemq/browse/CAMEL-1463
On Tue, Mar 17, 2009 at 9:15 AM, Claus Ibsen claus.ib...@gmail.com wrote:
On Fri, Mar 6, 2009 at 1:46 PM, _Jens jens.riemschnei...@icw.de wrote:
Hi,
first of all, thanks to both of you, Gert and Willem, for the quick replies.
I saw that the jira was already fixed, great job.
Does this mean that the methods for disabling stream caching will be removed
in the future?
I have tried to use noErrorHandler but there was a special case that used
the default error handler at some point and I wasn't able to change this. A
drilled down version of my scenario is shown in the test case below.
It has to do with the choice and when elements in the route. They seem to
introduce the standard error handler, as they don't inherit the configured
one. The standard error handling then configures the route to use stream
caching again. For some reason this only seems to happen if the JMX agent is
disabled in the CamelContext, which I simulate by setting the system
property. Of course, in our real scenario we simply disable it in the
application context.
So you are saying that with/without JMX it has an influence on your route?
That is kinda odd, but there could be a gremlin.
Need to check into this.
�...@test
public void testUploadFailsForBigFileWithNoErrorHandler() throws
Exception {
System.setProperty(JmxSystemPropertyKeys.DISABLED, true);
DefaultCamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
�...@override
public void configure() throws Exception {
errorHandler(noErrorHandler());
from(jetty:http://localhost:8989/bigfile;)
.choice().when(header(foo).isEqualTo(bar)).to(direct:end)
.otherwise().end();
from(direct:end)
.process(new Processor() {
public void process(Exchange exchange) throws
Exception {
InputStream inputStream =
exchange.getIn().getBody(InputStream.class);
while (inputStream.read() != -1) {}
}
});
}
});
context.start();
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(http://localhost:8989/bigfile;);
File file = new File(c:\\temp\\test.bin.ok2);
method.setRequestEntity(new FileRequestEntity(file,
unknown/unknown));
assertEquals(200, client.executeMethod(method));
}
--
View this message in context:
http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22371815.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: StreamCaching in Camel 1.6
Wow nice job! I'll make sure that I try 1.6.1 soon. Thanks a lot Claus. -- View this message in context: http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22581500.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: StreamCaching in Camel 1.6
On Fri, Mar 6, 2009 at 1:46 PM, _Jens jens.riemschnei...@icw.de wrote:
Hi,
first of all, thanks to both of you, Gert and Willem, for the quick replies.
I saw that the jira was already fixed, great job.
Does this mean that the methods for disabling stream caching will be removed
in the future?
I have tried to use noErrorHandler but there was a special case that used
the default error handler at some point and I wasn't able to change this. A
drilled down version of my scenario is shown in the test case below.
It has to do with the choice and when elements in the route. They seem to
introduce the standard error handler, as they don't inherit the configured
one. The standard error handling then configures the route to use stream
caching again. For some reason this only seems to happen if the JMX agent is
disabled in the CamelContext, which I simulate by setting the system
property. Of course, in our real scenario we simply disable it in the
application context.
So you are saying that with/without JMX it has an influence on your route?
That is kinda odd, but there could be a gremlin.
Need to check into this.
�...@test
public void testUploadFailsForBigFileWithNoErrorHandler() throws
Exception {
System.setProperty(JmxSystemPropertyKeys.DISABLED, true);
DefaultCamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
�...@override
public void configure() throws Exception {
errorHandler(noErrorHandler());
from(jetty:http://localhost:8989/bigfile;)
.choice().when(header(foo).isEqualTo(bar)).to(direct:end)
.otherwise().end();
from(direct:end)
.process(new Processor() {
public void process(Exchange exchange) throws
Exception {
InputStream inputStream =
exchange.getIn().getBody(InputStream.class);
while (inputStream.read() != -1) {}
}
});
}
});
context.start();
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(http://localhost:8989/bigfile;);
File file = new File(c:\\temp\\test.bin.ok2);
method.setRequestEntity(new FileRequestEntity(file,
unknown/unknown));
assertEquals(200, client.executeMethod(method));
}
--
View this message in context:
http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22371815.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: StreamCaching in Camel 1.6
Hi Claus, yes, it seems to have an effect because the lifecycle-strategy is different when disabling JMX. The Constructor of the DefaultCamelContext uses either DefaultLifecycleStrategy or InstrumentationLifecycleStrategy. This also has influence on the error handling strategy. Jens -- View this message in context: http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22554404.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: StreamCaching in Camel 1.6
Jens,
Stream caching is turned on automatically when you use a
DeadLetterChannel as the error handler. Because the Exchange might be
redelivered or sent to the DLQ destination, we have to cache it before
we send it into the DLC. Adding an errorHandler(noErrorHandler()); to
the configure() method should fix that for you.
Regards,
Gert Vanthienen
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/
2009/3/3 _Jens jens.riemschnei...@icw.de:
Hi,
I have run into a problem with stream caching in Camel 1.6. I have a route
that reads data via streaming from an Http request. In some cases this can
be a large amount of data, so I cannot stream the data completely into
memory. I wrote this little test to demonstrate my problem:
�...@test
public void testUploadFailsForHugeFile() throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
�...@override
public void configure() throws Exception {
noStreamCaching();
from(jetty:http://localhost:8989/bigfile;)
.noStreamCaching()
.process(new Processor() {
public void process(Exchange exchange) throws
Exception {
InputStream inputStream =
exchange.getIn().getBody(InputStream.class);
while (inputStream.read() != -1) {}
}
});
}
});
context.start();
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(http://localhost:8989/bigfile;);
File file = new File(c:\\temp\\test.bin.ok2);
method.setRequestEntity(new FileRequestEntity(file,
unknown/unknown));
assertEquals(200, client.executeMethod(method));
}
This fails with an OutOfMemoryError if test.bin.ok2 is a larger file because
the StreamCachingInterceptor reads the input stream into memory. Now I
understand that stream caching is turned on by default in Camel 1.6.
However, as you can see I tried to disable it in the route builder. This
seems not to make any difference.
Now, my question is if there is some other way to disable stream caching or
if I'm doing something wrong here.
Thanks in advance
Jens
--
View this message in context:
http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22305654.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: StreamCaching in Camel 1.6
Hi
Just as Gert said.
In CXF , we will cache the big message into the File instead of the memory.
I think we could do the same thing in Camel.
I just filled a JIRA[2] for it.
[1] http://www.nabble.com/Re%3A-StreamCaching-in-Camel-1.6-to22305858.html
[2] https://issues.apache.org/activemq/browse/CAMEL-1413
Willem
_Jens wrote:
Hi,
I have run into a problem with stream caching in Camel 1.6. I have a route
that reads data via streaming from an Http request. In some cases this can
be a large amount of data, so I cannot stream the data completely into
memory. I wrote this little test to demonstrate my problem:
@Test
public void testUploadFailsForHugeFile() throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
@Override
public void configure() throws Exception {
noStreamCaching();
from(jetty:http://localhost:8989/bigfile;)
.noStreamCaching()
.process(new Processor() {
public void process(Exchange exchange) throws
Exception {
InputStream inputStream =
exchange.getIn().getBody(InputStream.class);
while (inputStream.read() != -1) {}
}
});
}
});
context.start();
HttpClient client = new HttpClient();
PostMethod method = new
PostMethod(http://localhost:8989/bigfile;);
File file = new File(c:\\temp\\test.bin.ok2);
method.setRequestEntity(new FileRequestEntity(file,
unknown/unknown));
assertEquals(200, client.executeMethod(method));
}
This fails with an OutOfMemoryError if test.bin.ok2 is a larger file
because the StreamCachingInterceptor reads the input stream into memory.
Now I understand that stream caching is turned on by default in Camel 1.6.
However, as you can see I tried to disable it in the route builder. This
seems not to make any difference.
Now, my question is if there is some other way to disable stream caching
or if I'm doing something wrong here.
Thanks in advance
Jens
--
View this message in context:
http://www.nabble.com/StreamCaching-in-Camel-1.6-tp22305654p22306407.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
