Re: Delayer: Transport disposed at JVM-Shutdown
We maybe need to introduce some clean shutdown mechanism into consumers; so we can tell all the consumers we're about to shutdown; then they can gracefully stop processing the current route's message - then we actually stop the consumers (then the endpoints components etc). Stopping routes midway through (eg cancelling timers) could lead to other kinds of warning messages. I guess the downside of this could make shutdown take a while to complete - also if routes invoke other routes you still could have issues where one route wants to send to another route which is now closed. So its always gonna be pretty hard to shut down Camel cleanly without any error messages at all. Maybe we just hide error messages once shutdown starts? :) On 25/03/2009, Claus Ibsen claus.ib...@gmail.com wrote: On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Thanks, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22681944.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ -- James --- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/
Re: Delayer: Transport disposed at JVM-Shutdown
On Wed, Mar 25, 2009 at 10:02 AM, James Strachan james.strac...@gmail.com wrote: We maybe need to introduce some clean shutdown mechanism into consumers; so we can tell all the consumers we're about to shutdown; then they can gracefully stop processing the current route's message - then we actually stop the consumers (then the endpoints components etc). Stopping routes midway through (eg cancelling timers) could lead to other kinds of warning messages. I guess the downside of this could make shutdown take a while to complete - also if routes invoke other routes you still could have issues where one route wants to send to another route which is now closed. So its always gonna be pretty hard to shut down Camel cleanly without any error messages at all. Maybe we just hide error messages once shutdown starts? :) On 25/03/2009, Claus Ibsen claus.ib...@gmail.com wrote: On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Yeah I have given that some thoughts as well. I have wondered if we should support several shutdown strategies - shutdown now - shutdown - shutdown graceful Shutdown now, is a kinda abrubt shutdown. Shutdown is a kinda default shutdown as we do now And graceful: - Should deny any new incoming messages. - Schedulers should stop triggering - Exchanges in transit should allow to complete - And if Camel had a kinda of global state that could monitor if there were any ongoing exchanges. And thus wait until they completes before stopping all the routes. This global state monitor could however be a hot spot for really high performance. But I guess smart people have figured out great solutions. And this global monitor can be used in web console to see which in transit exchanges current exists. Thanks, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22681944.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ -- James --- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/ -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/
Re: Delayer: Transport disposed at JVM-Shutdown
2009/3/25 Claus Ibsen claus.ib...@gmail.com: On Wed, Mar 25, 2009 at 10:02 AM, James Strachan james.strac...@gmail.com wrote: We maybe need to introduce some clean shutdown mechanism into consumers; so we can tell all the consumers we're about to shutdown; then they can gracefully stop processing the current route's message - then we actually stop the consumers (then the endpoints components etc). Stopping routes midway through (eg cancelling timers) could lead to other kinds of warning messages. I guess the downside of this could make shutdown take a while to complete - also if routes invoke other routes you still could have issues where one route wants to send to another route which is now closed. So its always gonna be pretty hard to shut down Camel cleanly without any error messages at all. Maybe we just hide error messages once shutdown starts? :) On 25/03/2009, Claus Ibsen claus.ib...@gmail.com wrote: On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Yeah I have given that some thoughts as well. I have wondered if we should support several shutdown strategies - shutdown now - shutdown - shutdown graceful Shutdown now, is a kinda abrubt shutdown. Shutdown is a kinda default shutdown as we do now And graceful: - Should deny any new incoming messages. - Schedulers should stop triggering - Exchanges in transit should allow to complete - And if Camel had a kinda of global state that could monitor if there were any ongoing exchanges. And thus wait until they completes before stopping all the routes. This global state monitor could however be a hot spot for really high performance. But I guess smart people have figured out great solutions. And this global monitor can be used in web console to see which in transit exchanges current exists. Agreed. Right now DefaultCamelContext has isStopped() and isStopping() methods which consumers could look for; maybe we need to expose some public method which returns true if a consumer should continue processing (and if it returns false then they should gracefully shut down). e.g. we add some method like this to CamelContext API /** Returns true if a consumer can continue processing or false if it should gracefully stop */ boolean isConsumeAllowed(); Things like Guice/Spring have lifecycle support so will shut down the CamelContext and any related services; so maybe we just need some gracefulShutdown flag we can enable - which causes the doStop() in DefaultCamelContext to wait for all the consumers to be stopped before closing any endpoints/producers/services? To implement this we might need to register all consumers in the CamelContext until they are stopped maybe... -- James --- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/
Re: Delayer: Transport disposed at JVM-Shutdown
On Wed, Mar 25, 2009 at 10:49 AM, James Strachan james.strac...@gmail.com wrote: 2009/3/25 Claus Ibsen claus.ib...@gmail.com: On Wed, Mar 25, 2009 at 10:02 AM, James Strachan james.strac...@gmail.com wrote: We maybe need to introduce some clean shutdown mechanism into consumers; so we can tell all the consumers we're about to shutdown; then they can gracefully stop processing the current route's message - then we actually stop the consumers (then the endpoints components etc). Stopping routes midway through (eg cancelling timers) could lead to other kinds of warning messages. I guess the downside of this could make shutdown take a while to complete - also if routes invoke other routes you still could have issues where one route wants to send to another route which is now closed. So its always gonna be pretty hard to shut down Camel cleanly without any error messages at all. Maybe we just hide error messages once shutdown starts? :) On 25/03/2009, Claus Ibsen claus.ib...@gmail.com wrote: On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Yeah I have given that some thoughts as well. I have wondered if we should support several shutdown strategies - shutdown now - shutdown - shutdown graceful Shutdown now, is a kinda abrubt shutdown. Shutdown is a kinda default shutdown as we do now And graceful: - Should deny any new incoming messages. - Schedulers should stop triggering - Exchanges in transit should allow to complete - And if Camel had a kinda of global state that could monitor if there were any ongoing exchanges. And thus wait until they completes before stopping all the routes. This global state monitor could however be a hot spot for really high performance. But I guess smart people have figured out great solutions. And this global monitor can be used in web console to see which in transit exchanges current exists. Agreed. Right now DefaultCamelContext has isStopped() and isStopping() methods which consumers could look for; maybe we need to expose some public method which returns true if a consumer should continue processing (and if it returns false then they should gracefully shut down). e.g. we add some method like this to CamelContext API /** Returns true if a consumer can continue processing or false if it should gracefully stop */ boolean isConsumeAllowed(); Well what if a consumer is part of an internal route as well. eg the seda component can be used to bind routes together. So if we stop the seda consumer and there are some routes active with messages being routed to this consumer? They will fail then. I guess its hard to stop any outside bound consumers/messages coming in. Things like Guice/Spring have lifecycle support so will shut down the CamelContext and any related services; so maybe we just need some gracefulShutdown flag we can enable - which causes the doStop() in DefaultCamelContext to wait for all the consumers to be stopped before closing any endpoints/producers/services? To implement this we might need to register all consumers in the CamelContext until they are stopped maybe... -- James --- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/ -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/
Re: Delayer: Transport disposed at JVM-Shutdown
Hi Created a ticket for better graceful shutdown in Camel https://issues.apache.org/activemq/browse/CAMEL-1483 On Wed, Mar 25, 2009 at 11:06 AM, Claus Ibsen claus.ib...@gmail.com wrote: On Wed, Mar 25, 2009 at 10:49 AM, James Strachan james.strac...@gmail.com wrote: 2009/3/25 Claus Ibsen claus.ib...@gmail.com: On Wed, Mar 25, 2009 at 10:02 AM, James Strachan james.strac...@gmail.com wrote: We maybe need to introduce some clean shutdown mechanism into consumers; so we can tell all the consumers we're about to shutdown; then they can gracefully stop processing the current route's message - then we actually stop the consumers (then the endpoints components etc). Stopping routes midway through (eg cancelling timers) could lead to other kinds of warning messages. I guess the downside of this could make shutdown take a while to complete - also if routes invoke other routes you still could have issues where one route wants to send to another route which is now closed. So its always gonna be pretty hard to shut down Camel cleanly without any error messages at all. Maybe we just hide error messages once shutdown starts? :) On 25/03/2009, Claus Ibsen claus.ib...@gmail.com wrote: On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Yeah I have given that some thoughts as well. I have wondered if we should support several shutdown strategies - shutdown now - shutdown - shutdown graceful Shutdown now, is a kinda abrubt shutdown. Shutdown is a kinda default shutdown as we do now And graceful: - Should deny any new incoming messages. - Schedulers should stop triggering - Exchanges in transit should allow to complete - And if Camel had a kinda of global state that could monitor if there were any ongoing exchanges. And thus wait until they completes before stopping all the routes. This global state monitor could however be a hot spot for really high performance. But I guess smart people have figured out great solutions. And this global monitor can be used in web console to see which in transit exchanges current exists. Agreed. Right now DefaultCamelContext has isStopped() and isStopping() methods which consumers could look for; maybe we need to expose some public method which returns true if a consumer should continue processing (and if it returns false then they should gracefully shut down). e.g. we add some method like this to CamelContext API /** Returns true if a consumer can continue processing or false if it should gracefully stop */ boolean isConsumeAllowed(); Well what if a consumer is part of an internal route as well. eg the seda component can be used to bind routes together. So if we stop the seda consumer and there are some routes active with messages being routed to this consumer? They will fail then. I guess its hard to stop any outside bound consumers/messages coming in. Things like Guice/Spring have lifecycle support so will shut down the CamelContext and any related services; so maybe we just need some gracefulShutdown flag we can enable - which causes the doStop() in DefaultCamelContext to wait for all the consumers to be stopped before closing any endpoints/producers/services? To implement this we might need to register all consumers in the CamelContext until they are stopped maybe... -- James --- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/ -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
Re: Delayer: Transport disposed at JVM-Shutdown
Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Thanks, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22681944.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: Delayer: Transport disposed at JVM-Shutdown
On Tue, Mar 24, 2009 at 3:42 PM, Markus Reil gistenju...@gmx.de wrote: Hi, thanks for your help. But I am still experiencing the same problem. The sleep does not get interrupted (2.0-M1). Do you have any idea why this happens? Let me know if you need more information. Can you show the complete route? And point out where the message is when you shutdown. Thanks, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22681944.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/
Re: Delayer: Transport disposed at JVM-Shutdown
Hi, thanks for your reply. Claus Ibsen-2 wrote: If you do NOT way any error handling in camel just set the error handler to no error handler :) Yeah maybe we need a nicer DSL for disabling it errorHandler(noErrorHandler()); That works. Thanks. Claus Ibsen-2 wrote: Setting up a custom ExceptionListener did not change anything. What do you mean? A onException or what? Right. I implemented ExceptionListener.onException() and set it as the JmsComponent's exceptionListener. Do you have an idea for my original problem? I only get the Transport disposed error beause the messages are delayed. It seems like the sleep doesn't get interrupted on shutdown. Thanks, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22537092.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: Delayer: Transport disposed at JVM-Shutdown
Claus Ibsen-2 wrote:
Which version of Camel are you using?
And are you sure there is a message in delay in Camel, while you shutdown?
Are you also using JMS?
I'm using 1.5.0 at the moment and I'm sure the message is in delay:
[15:27:03.598] [DEBUG] [DelayRoutes] Save exchange to wait queue
I shutdown here...
[15:27:33.656] [DEBUG] [DelayRoutes] Message will be sent to target queue
(activemq:queue:queue.profileComputation.out).
And yes, I am using JMS.
I just tried apache-camel-2.0-20090311.162311-1.tar.gz and I still don't see
the Interrupted log message. I do not get the Transport disposed
exception anymore but the server won't shutdown at all. Last log lines are:
[15:27:10.613] [ INFO] [org.apache.camel.impl.DefaultCamelContext] Apache
Camel (CamelContext:null) is stopping
[15:27:33.655] [DEBUG] [com.kekaya.activemq.DelayRoutes] headers:
properties:{CamelBeanHolder=Routes: [], kekayaExecTime=1237213653596}
headers:{JMSCorrelationID=null,
JMSMessageID=ID:localhost-59869-1237213501544-2:0:1:1:1, JMSExpiration=0,
kekayaDelay=30, kekayaPass=2, JMSDeliveryMode=2,
kekayaExecTime=1237213653596, JMSPriority=4, kekayaId=3,
JMSTimestamp=1237213623616, JMSReplyTo=null,
JMSDestination=queue://queue.profileComputation.wait, JMSXGroupID=null,
JMSType=null, JMSRedelivered=false}
[15:27:33.656] [DEBUG] [com.kekaya.activemq.DelayRoutes] Message will be
sent to target queue (activemq:queue:queue.profileComputation.out).
[15:27:33.657] [DEBUG] [org.apache.camel.impl.DefaultComponent] Creating
endpoint uri=[activemq:queue:queue.profileComputation.out],
path=[queue:queue.profileComputation.out], parameters=[{}]
[15:27:33.657] [DEBUG] [org.apache.camel.impl.DefaultCamelContext] []
activemq:queue:queue.profileComputation.out converted to endpoint:
Endpoint[activemq:queue:queue.profileComputation.out] by component:
org.apache.camel.component.jms.jmscompon...@1eb2e9f
--
View this message in context:
http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22539534.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Re: Delayer: Transport disposed at JVM-Shutdown
On Mon, Mar 16, 2009 at 3:34 PM, Markus Reil gistenju...@gmx.de wrote:
Claus Ibsen-2 wrote:
Which version of Camel are you using?
And are you sure there is a message in delay in Camel, while you shutdown?
Are you also using JMS?
I'm using 1.5.0 at the moment and I'm sure the message is in delay:
[15:27:03.598] [DEBUG] [DelayRoutes] Save exchange to wait queue
I shutdown here...
[15:27:33.656] [DEBUG] [DelayRoutes] Message will be sent to target queue
(activemq:queue:queue.profileComputation.out).
And yes, I am using JMS.
The fix is in 2.0-SNAPSHOT. Or the 2.0M1 that is published on the
central maven repo later today.
I just tried apache-camel-2.0-20090311.162311-1.tar.gz and I still don't see
the Interrupted log message. I do not get the Transport disposed
exception anymore but the server won't shutdown at all. Last log lines are:
[15:27:10.613] [ INFO] [org.apache.camel.impl.DefaultCamelContext] Apache
Camel (CamelContext:null) is stopping
[15:27:33.655] [DEBUG] [com.kekaya.activemq.DelayRoutes] headers:
properties:{CamelBeanHolder=Routes: [], kekayaExecTime=1237213653596}
headers:{JMSCorrelationID=null,
JMSMessageID=ID:localhost-59869-1237213501544-2:0:1:1:1, JMSExpiration=0,
kekayaDelay=30, kekayaPass=2, JMSDeliveryMode=2,
kekayaExecTime=1237213653596, JMSPriority=4, kekayaId=3,
JMSTimestamp=1237213623616, JMSReplyTo=null,
JMSDestination=queue://queue.profileComputation.wait, JMSXGroupID=null,
JMSType=null, JMSRedelivered=false}
[15:27:33.656] [DEBUG] [com.kekaya.activemq.DelayRoutes] Message will be
sent to target queue (activemq:queue:queue.profileComputation.out).
[15:27:33.657] [DEBUG] [org.apache.camel.impl.DefaultComponent] Creating
endpoint uri=[activemq:queue:queue.profileComputation.out],
path=[queue:queue.profileComputation.out], parameters=[{}]
[15:27:33.657] [DEBUG] [org.apache.camel.impl.DefaultCamelContext] []
activemq:queue:queue.profileComputation.out converted to endpoint:
Endpoint[activemq:queue:queue.profileComputation.out] by component:
org.apache.camel.component.jms.jmscompon...@1eb2e9f
--
View this message in context:
http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22539534.html
Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: Delayer: Transport disposed at JVM-Shutdown
On Fri, Mar 13, 2009 at 5:56 PM, Markus Reil gistenju...@gmx.de wrote: Hi, does anyone have an idea here please? I tried to configure a global errorHandler like errorHandler(loggingErrorHandler(com.kekaya.activemq)); or errorHandler(deadLetterChannel(direct:err).maximumRedeliveries(0)); but the Exception still gets processed by camel's default DeadLetterChannel. If you do NOT way any error handling in camel just set the error handler to no error handler :) Yeah maybe we need a nicer DSL for disabling it errorHandler(noErrorHandler()); Setting up a custom ExceptionListener did not change anything. What do you mean? A onException or what? Any help is appreciated. Cheers, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22500700.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/
Re: Delayer: Transport disposed at JVM-Shutdown
Hi, does anyone have an idea here please? I tried to configure a global errorHandler like errorHandler(loggingErrorHandler(com.kekaya.activemq)); or errorHandler(deadLetterChannel(direct:err).maximumRedeliveries(0)); but the Exception still gets processed by camel's default DeadLetterChannel. Setting up a custom ExceptionListener did not change anything. Any help is appreciated. Cheers, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22500700.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
Delayer: Transport disposed at JVM-Shutdown
Hi, I frequently get a Transport disposed error for delayed messages when I shutdown Tomcat. My route is configured like this: from(queue).delayer(3L).bean(customRouterBean).recipientList(HEADER_TARGET_ROUTE); Depending on some internal state my customRouterBean decides where the message goes. When I shutdown Tomcat while a message is being delayed the router cannot deliver the message finally because a queue (or the broker?) isn't available anymore. Can I find out in the router if a queue is still usable? And is it possible to interrupt the delay at this point because delaying the message also delays the server shutdown? Thanks in advance for any help. Cheers, Markus -- View this message in context: http://www.nabble.com/Delayer%3A-%22Transport-disposed%22-at-JVM-Shutdown-tp22202167p22202167.html Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
