[
https://issues.apache.org/jira/browse/QPID-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633377#comment-14633377
]
Lorenz Quack commented on QPID-6649:
------------------------------------
* move NonBlockingConnection#shutdownOutput into a finally block
* reorganise the way deregisterConnection works so that the connection stays
associated with the VHost until it the socket has closed
* make the futures of asyncClose of connections accessible to both protocol and
model layer so that when either party closes a connection the other party can
await the closure
> Client initiated connection close can race with Broker shutdown (or VH
> mastership change) leading to exception and unclosed socket
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: QPID-6649
> URL: https://issues.apache.org/jira/browse/QPID-6649
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Reporter: Keith Wall
> Assignee: Keith Wall
> Fix For: 6.0 [Java]
>
> Attachments:
> 0001-QPID-6649-Ensure-socket-is-closed.-Make-QBTC-fail-if.patch
>
>
> There is a potential for a race in the Java Broker that is being exposed by
> the System Tests. The unlucky case is as follows. The client closes the
> connection by sending ConnectionClose. On receiving this,
> ServerConnectionDelegate.connectionClose() invokes
> ServerConnection.setState() which invokes
> AbstractVirtualHost.deregisterConnectionAsync(). This causes the AVH to
> 'forget' the connection (removed from AVH#_connections) even though it is not
> yet closed. Meanwhile, QBTC begins to close the Broker. The Broker closes
> the AVH which calls closeConnections - but the connection is already for
> forgotten (_connections#size() == 0) so as far as it is concerned there are
> no connections to await for, so the VH close concludes and the VHN shutdowns
> the VH's executor. Then the IO thread gets scheduled, sends the
> ConnectionCloseOk bytes down the wire and then NBC#shutdown invokes
> MVPE#close.... which finally invokesServerConnection.setState which calls
> AbstractVirtualHost.deregisterConnectionAsync, which fails (stack trace
> below) because the task executor is already closed.
> This causes a socket to remain open.
> {noformat}
> java.lang.IllegalStateException: Task executor is not in ACTIVE state
> at
> org.apache.qpid.server.configuration.updater.TaskExecutorImpl.checkState(TaskExecutorImpl.java:310)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.configuration.updater.TaskExecutorImpl.submit(TaskExecutorImpl.java:141)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.model.AbstractConfiguredObject.doOnConfigThread(AbstractConfiguredObject.java:499)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.virtualhost.AbstractVirtualHost.deregisterConnectionAsync(AbstractVirtualHost.java:1753)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.virtualhost.AbstractVirtualHost.deregisterConnection(AbstractVirtualHost.java:1748)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_10.ServerConnection.setState(ServerConnection.java:161)
> ~[qpid-broker-plugins-amqp-0-10-protocol-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at org.apache.qpid.transport.Connection.closed(Connection.java:594)
> ~[qpid-common-6.0.0-SNAPSHOT.jar:na]
> at
> org.apache.qpid.server.protocol.v0_10.ServerConnection.closed(ServerConnection.java:495)
> ~[qpid-broker-plugins-amqp-0-10-protocol-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.transport.network.Assembler.closed(Assembler.java:113)
> ~[qpid-common-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.transport.network.InputHandler.closed(InputHandler.java:215)
> ~[qpid-common-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.protocol.v0_10.AMQPConnection_0_10.closed(AMQPConnection_0_10.java:259)
> ~[qpid-broker-plugins-amqp-0-10-protocol-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.MultiVersionProtocolEngine.closed(MultiVersionProtocolEngine.java:108)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NonBlockingConnection.shutdown(NonBlockingConnection.java:268)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:255)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:119)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NetworkConnectionScheduler.access$000(NetworkConnectionScheduler.java:37)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> org.apache.qpid.server.transport.NetworkConnectionScheduler$2.run(NetworkConnectionScheduler.java:100)
> ~[qpid-broker-core-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_65]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
