Hi users,
we've deployed ActiveMQ 5.7 on several remote hosts and configured a
static broker network between the remote nodes and a central broker in
our datacenter according to [1] (typical star architecture).
The setup seems to be working fine, messages enqueued on the remote
brokers get forwarded to the central broker via the network bridge.
If the underlying WAN connection is interrupted for some reason, the
bridge is able to recover the connection to the central broker most of
the time.
Sometimes though the remote broker can't recover the connection and
messages are starting to pile up. Around the same time the last message
is forwarded to the central broker, the log file of the remote broker
shows that it's trying to establish the network bridge with the central
broker.
This attempt fails with the message: "Failed to add static destination
queue://queue1" and a TransportDisposedIOException (see [3] for the log
entry).
From this moment on the network bridge to the central broker won't be
established again. It seems that this situation can only be resolved
by restarting the remote broker process.
Has anybody experienced this behavior before with unreliable WAN
connections?
Any help would be appreciated.
Chris
[1] http://activemq.apache.org/networks-of-brokers.html
[2] Configuration of network bridge on the remote broker
<networkConnectors>
<networkConnector
uri="static:(tcp://centralbroker:61616)"
staticBridge="true"
name="bridgeToDataCenter"
duplex="false"
conduitSubscriptions="true" >
<staticallyIncludedDestinations>
<queue physicalName="queue1" />
</staticallyIncludedDestinations>
</networkConnector>
</networkConnectors>
[3] activemq.log
2013-05-22 17:30:30,847 | INFO | il00347#94 |
mq.network.DemandForwardingBridgeSupport | Network connection between
vm://remotebroker#94 and
tcp://centralbroker/1.2.3.4:61616@42764(centralbroker) has been established.
2013-05-22 17:30:30,913 | WARN | 1616@42764 |
mq.network.DemandForwardingBridgeSupport | Network connection between
vm://remotebroker#94 and tcp://centralbroker/1.2.3.4:61616@42764
shutdown due to a remote error: javax.jms.InvalidClientIDException:
Broker: centralbroker - Client:
remotebroker-queues_remotebroker_outbound already connected from
tcp://4.3.2.1:42202
2013-05-22 17:30:31,021 | INFO | 7] Task-78 |
mq.network.DemandForwardingBridgeSupport | remotebroker bridge to
centralbroker stopped
2013-05-22 17:30:31,023 | ERROR | il00347#94 |
mq.network.DemandForwardingBridgeSupport | Failed to add static
destination queue://queue1
org.apache.activemq.transport.TransportDisposedIOException: Peer
(vm://remotebroker#95) disposed.
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:89)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.addSubscription(DemandForwardingBridgeSupport.java:649)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.setupStaticDestinations(DemandForwardingBridgeSupport.java:981)
at
org.apache.activemq.network.DurableConduitBridge.setupStaticDestinations(DurableConduitBridge.java:53)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startLocalBridge(DemandForwardingBridgeSupport.java:281)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.access$000(DemandForwardingBridgeSupport.java:92)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$3.run(DemandForwardingBridgeSupport.java:209)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
