[
https://issues.apache.org/jira/browse/AMQ-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Minearo updated AMQ-5262:
-------------------------------
Description:
We are having a problem with ActiveMQ hanging on shutdown. Here is the
scenario, we have a stand alone application that runs an embedded ActiveMQ
which creates a JMS Queue Bridge via Spring configs. When we call shutdown, the
TCPTransport that connects the JMS Queue Bridge does not shutdown, it hangs on
java.net.SocketInputStream.socketRead0().
{noformat}
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
at
org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:604)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
at
org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:589)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:662)
{noformat}
Digging around on the forums and the issue tracker, the work around seems to
add a parameter to the URI (Ex - tcp://localhost:60606?daemon=true).
According to this StackOverflow posting
(http://stackoverflow.com/questions/2213340/what-is-daemon-thread-in-java)
which quotes from Java Concurrency in Practice
* When a new thread is created it inherits the daemon status of its parent.
* Normal thread and daemon threads differ in what happens when they exit. When
the JVM halts any remaining daemon threads are abandoned: **finally blocks are
not executed**, stacks are not unwound - JVM just exits. Due to this reason
daemon threads should be used sparingly and it is dangerous to use them for
tasks that might perform any sort of I/O.
So, making the Socket that connects the JMS Queue Bridge a Daemon thread, seems
to be the wrong solution.
I was trying to debug the initialization of ActiveMQ, and noticed the
org.apache.activemq.network.jms.JmsConnector class has a stop() method on it.
I believe this class creates the connection for the JMS Bridge, right? If so,
the stop method does not seem to shutdown the connection properly.
{code}
public void stop() throws Exception {
if (started.compareAndSet(true, false)) {
ThreadPoolUtils.shutdown(connectionSerivce);
connectionSerivce = null;
for (DestinationBridge bridge : inboundBridges) {
bridge.stop();
}
for (DestinationBridge bridge : outboundBridges) {
bridge.stop();
}
LOG.info("JMS Connector {} stopped", getName());
}
}
{code}
The question I have is why is the stop() method relying on the
ThreadPoolUtils.shutdown(connectionSerivce) and NOT calling close() on the
Connections first? For example:
{code}
public void stop() throws Exception {
if (started.compareAndSet(true, false)) {
foreignConnection.get().close();
localConnection.get().close();
ThreadPoolUtils.shutdown(connectionSerivce);
connectionSerivce = null;
for (DestinationBridge bridge : inboundBridges) {
bridge.stop();
}
for (DestinationBridge bridge : outboundBridges) {
bridge.stop();
}
LOG.info("JMS Connector {} stopped", getName());
}
}
{code}
It was a little difficult to follow the code, so I may be missing something.
BUT shouldn't the connections close first? Or am I looking in the wrong place.
I have created a small project that creates this scenario.
https://github.com/pminearo/activemq-shutdown-bug.git
This was done with ActiveMQ 5.9. Though, since this bug has been around for
quite some time, it most likely is still in 5.10, 5.11, and 6.0.
was:
We are having a problem with ActiveMQ hanging on shutdown. Here is the
scenario, we have a stand alone application that runs an embedded ActiveMQ
which creates a JMS Queue Bridge via Spring configs. When we call shutdown, the
TCPTransport that connects the JMS Queue Bridge does not shutdown, it hangs on
java.net.SocketInputStream.socketRead0().
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
at
org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:604)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
at
org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:589)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:662)
Digging around on the forums and the issue tracker, the work around seems to
add a parameter to the URI (Ex - tcp://localhost:60606?daemon=true).
According to this StackOverflow posting
(http://stackoverflow.com/questions/2213340/what-is-daemon-thread-in-java)
which quotes from Java Concurrency in Practice
* When a new thread is created it inherits the daemon status of its parent.
* Normal thread and daemon threads differ in what happens when they exit. When
the JVM halts any remaining daemon threads are abandoned: **finally blocks are
not executed**, stacks are not unwound - JVM just exits. Due to this reason
daemon threads should be used sparingly and it is dangerous to use them for
tasks that might perform any sort of I/O.
So, making the Socket that connects the JMS Queue Bridge a Daemon thread, seems
to be the wrong solution.
I was trying to debug the initialization of ActiveMQ, and noticed the
org.apache.activemq.network.jms.JmsConnector class has a stop() method on it.
I believe this class creates the connection for the JMS Bridge, right? If so,
the stop method does not seem to shutdown the connection properly.
public void stop() throws Exception {
if (started.compareAndSet(true, false)) {
ThreadPoolUtils.shutdown(connectionSerivce);
connectionSerivce = null;
for (DestinationBridge bridge : inboundBridges) {
bridge.stop();
}
for (DestinationBridge bridge : outboundBridges) {
bridge.stop();
}
LOG.info("JMS Connector {} stopped", getName());
}
}
The question I have is why is the stop() method relying on the
ThreadPoolUtils.shutdown(connectionSerivce) and NOT calling close() on the
Connections first? For example:
public void stop() throws Exception {
if (started.compareAndSet(true, false)) {
foreignConnection.get().close();
localConnection.get().close();
ThreadPoolUtils.shutdown(connectionSerivce);
connectionSerivce = null;
for (DestinationBridge bridge : inboundBridges) {
bridge.stop();
}
for (DestinationBridge bridge : outboundBridges) {
bridge.stop();
}
LOG.info("JMS Connector {} stopped", getName());
}
}
It was a little difficult to follow the code, so I may be missing something.
BUT shouldn't the connections close first? Or am I looking in the wrong place.
I have created a small project that creates this scenario.
https://github.com/pminearo/activemq-shutdown-bug.git
This was done with ActiveMQ 5.9. Though, since this bug has been around for
quite some time, it most likely is still in 5.10, 5.11, and 6.0.
> ActiveMQ hangs on shutdown when JMS Bridge is created
> -----------------------------------------------------
>
> Key: AMQ-5262
> URL: https://issues.apache.org/jira/browse/AMQ-5262
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.9.0, 5.9.1, 5.10.0
> Reporter: Peter Minearo
> Priority: Minor
>
> We are having a problem with ActiveMQ hanging on shutdown. Here is the
> scenario, we have a stand alone application that runs an embedded ActiveMQ
> which creates a JMS Queue Bridge via Spring configs. When we call shutdown,
> the TCPTransport that connects the JMS Queue Bridge does not shutdown, it
> hangs on java.net.SocketInputStream.socketRead0().
> {noformat}
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at
> org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
> at
> org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:604)
> at
> org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
> at
> org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:589)
> at java.io.DataInputStream.readInt(DataInputStream.java:370)
> at
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
> at
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> Digging around on the forums and the issue tracker, the work around seems to
> add a parameter to the URI (Ex - tcp://localhost:60606?daemon=true).
> According to this StackOverflow posting
> (http://stackoverflow.com/questions/2213340/what-is-daemon-thread-in-java)
> which quotes from Java Concurrency in Practice
> * When a new thread is created it inherits the daemon status of its parent.
> * Normal thread and daemon threads differ in what happens when they exit.
> When the JVM halts any remaining daemon threads are abandoned: **finally
> blocks are not executed**, stacks are not unwound - JVM just exits. Due to
> this reason daemon threads should be used sparingly and it is dangerous to
> use them for tasks that might perform any sort of I/O.
> So, making the Socket that connects the JMS Queue Bridge a Daemon thread,
> seems to be the wrong solution.
> I was trying to debug the initialization of ActiveMQ, and noticed the
> org.apache.activemq.network.jms.JmsConnector class has a stop() method on it.
> I believe this class creates the connection for the JMS Bridge, right? If
> so, the stop method does not seem to shutdown the connection properly.
> {code}
> public void stop() throws Exception {
> if (started.compareAndSet(true, false)) {
> ThreadPoolUtils.shutdown(connectionSerivce);
> connectionSerivce = null;
> for (DestinationBridge bridge : inboundBridges) {
> bridge.stop();
> }
> for (DestinationBridge bridge : outboundBridges) {
> bridge.stop();
> }
> LOG.info("JMS Connector {} stopped", getName());
> }
>
> }
> {code}
> The question I have is why is the stop() method relying on the
> ThreadPoolUtils.shutdown(connectionSerivce) and NOT calling close() on the
> Connections first? For example:
> {code}
> public void stop() throws Exception {
> if (started.compareAndSet(true, false)) {
> foreignConnection.get().close();
> localConnection.get().close();
>
> ThreadPoolUtils.shutdown(connectionSerivce);
> connectionSerivce = null;
> for (DestinationBridge bridge : inboundBridges) {
> bridge.stop();
> }
> for (DestinationBridge bridge : outboundBridges) {
> bridge.stop();
> }
> LOG.info("JMS Connector {} stopped", getName());
> }
>
> }
> {code}
> It was a little difficult to follow the code, so I may be missing something.
> BUT shouldn't the connections close first? Or am I looking in the wrong
> place.
> I have created a small project that creates this scenario.
> https://github.com/pminearo/activemq-shutdown-bug.git
> This was done with ActiveMQ 5.9. Though, since this bug has been around for
> quite some time, it most likely is still in 5.10, 5.11, and 6.0.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
