[
https://issues.apache.org/activemq/browse/AMQ-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Snyder updated AMQ-1352:
------------------------------
Fix Version/s: (was: AGING_TO_DIE)
5.4.0
> Shutdown fails if DB connection is lost
> ---------------------------------------
>
> Key: AMQ-1352
> URL: https://issues.apache.org/activemq/browse/AMQ-1352
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.0.0
> Reporter: Mario Siegenthaler
> Fix For: 5.4.0
>
>
> The broker does not shutdown cleanly if the db connection for the JDBC
> persistence gets disconnected.
> I discovered this issue when implementing the fix for the AMQ-1350. When the
> broker discovers it doesn't hold the db-lock anymore (because the connection
> is lost, killed via the db-administration tool) it initiates its shutdown.
> This shutdown fails most of the time, especially if there is a camel-rule or
> a webconsole started within the broker.
> The thing is that the broker actually has been shut down (as it's stated and
> intended), but the other things (jetty, commandAgent, camel etc.) will not be
> shut down and the java-process does not terminate.
> IMO the broker shutdown didn't even fail, it's just issued an
> InterruptedException somewhere in order to stop some connector thread. I
> think this just needs a proper exception handling/ignoring.
> The brokers console output:
> INFO JDBCPersistenceAdapter - No longer able to keep the exclusive
> lock so giving up being a master
> INFO BrokerService - ActiveMQ Message Broker
> (marios-master,
> ID:mario-siegenthalers-computer.local-64808-1185663859419-1:0) is shutting
> down
> WARN ActiveMQConnection - Async exception with no exception
> listener: java.io.EOFException
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:356)
> at
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269)
> at
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:156)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:136)
> at java.lang.Thread.run(Thread.java:637)
> INFO AdvisoryConsumer - Failed to send remove command:
> javax.jms.JMSException: The transport is not running.
> javax.jms.JMSException: The transport is not running.
> at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:58)
> at
> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1205)
> at
> org.apache.activemq.AdvisoryConsumer.dispose(AdvisoryConsumer.java:55)
> at
> org.apache.activemq.ActiveMQConnection.cleanup(ActiveMQConnection.java:1378)
> at
> org.apache.activemq.ActiveMQConnection.transportFailed(ActiveMQConnection.java:2074)
> at
> org.apache.activemq.ActiveMQConnection$4.run(ActiveMQConnection.java:1708)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: java.io.IOException: The transport is not running.
> at
> org.apache.activemq.transport.TransportSupport.checkStarted(TransportSupport.java:104)
> at
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:117)
> at
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:147)
> at
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:80)
> at
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:93)
> at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
> at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
> at
> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1203)
> ... 7 more
> INFO TransportConnector - Connector openwire Stopped
> ERROR efaultMessageListenerContainer - Setup of JMS message listener invoker
> failed - trying to recover
> javax.jms.IllegalStateException: The Consumer is closed
> at
> org.apache.activemq.ActiveMQMessageConsumer.checkClosed(ActiveMQMessageConsumer.java:650)
> at
> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:484)
> at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:375)
> at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:300)
> at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:254)
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:871)
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:818)
> at java.lang.Thread.run(Thread.java:637)
> ERROR ManagedQueueRegion - Could not stop service: QueueRegion:
> destinations=1, subscriptions=0, memory=0%. Reason:
> java.lang.InterruptedException
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at
> org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at
> org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at
> org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at
> org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at
> org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> ERROR BrokerService$2 - Could not stop service:
> org.apache.activemq.broker.brokerservic...@b59fd9. Reason:
> java.lang.InterruptedException
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at
> org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at
> org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at
> org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at
> org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at
> org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> INFO BrokerService - ActiveMQ JMS Message Broker
> (marios-master,
> ID:mario-siegenthalers-computer.local-64808-1185663859419-1:0) stopped
> WARN JDBCPersistenceAdapter - Failed to stop broker
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at
> org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at
> org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at
> org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at
> org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at
> org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at
> org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at
> org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at
> org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
