Tanky Fu created AMQ-6814:
-----------------------------
Summary: Network broker Async error
Key: AMQ-6814
URL: https://issues.apache.org/jira/browse/AMQ-6814
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.12.1, 5.11.1
Environment: centos 6.3/6.5 activemq 5.11.1
Reporter: Tanky Fu
Fix For: Unscheduled
Dear:
we use activemq 5.11.1 network broker, it has three bokers,we find that
when the persistencedata is big,we use the topic and queue is large ,one of the
network broker will happened this warn:WARN | Async error occurred and then
it will reconnect to the other many times,the log as follows:
---------------------------------------------------------------------------
2017-09-08 13:58:13,136 | WARN | Async error occurred: |
org.apache.activemq.broker.TransportConnection.Service | pool-35-thread-1
java.lang.IllegalStateException: Cannot remove a consumer that had not been
registered: dynamic-broker1->dynamic-broker3-46135-1502950545007-818:1:1:29309
at
org.apache.activemq.broker.TransportConnection.processRemoveConsumer(TransportConnection.java:691)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:76)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:130)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:107)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$10.run(DemandForwardingBridgeSupport.java:934)[activemq-broker-5.11.1.jar:5.11.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2017-09-08 13:58:13,138 | INFO | Network connection between
vm://dynamic-broker1#68 and tcp:///192.168.1.138:61620@47627 shutdown due to a
local error: java.lang.IllegalStateException: Cannot remove a consumer that had
not been registered:
dynamic-broker1->dynamic-broker3-46135-1502950545007-818:1:1:29309 |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[dynamic-broker1] Task-161016
2017-09-08 13:58:13,142 | INFO | Establishing network connection from
vm://dynamic-broker1?async=false&network=true to tcp://192.168.1.138:61620 |
org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-12
2017-09-08 13:58:13,175 | WARN | Failed to add Connection
dynamic-broker1->dynamic-broker3-46135-1502950545007-1231:1 |
org.apache.activemq.broker.TransportConnection |
triggerStartAsyncNetworkBridgeCreation:
remoteBroker=tcp:///192.168.1.138:61620@50765, localBroker=
vm://dynamic-broker1#84
javax.jms.InvalidClientIDException: Broker: dynamic-broker1 - Client:
NC_dynamic-broker3_inbound_dynamic-broker1 already connected from
vm://dynamic-broker1#68
at
org.apache.activemq.broker.region.RegionBroker.addConnection(RegionBroker.java:255)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.jmx.ManagedRegionBroker.addConnection(ManagedRegionBroker.java:231)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:92)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:102)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:809)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:130)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:107)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startLocalBridge(DemandForwardingBridgeSupport.java:467)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.doStartLocalAndRemoteBridges(DemandForwardingBridgeSupport.java:420)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.access$500(DemandForwardingBridgeSupport.java:105)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$5.run(DemandForwardingBridgeSupport.java:335)[activemq-broker-5.11.1.jar:5.11.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2017-09-08 13:58:13,193 | INFO | Network connection between
vm://dynamic-broker1#84 and tcp:///192.168.1.138:61620@50765 shutdown due to a
local error: javax.jms.InvalidClientIDException: Broker: dynamic-broker1 -
Client: NC_dynamic-broker3_inbound_dynamic-broker1 already connected from
vm://dynamic-broker1#68 |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation:
remoteBroker=tcp:///192.168.1.138:61620@50765, localBroker=
vm://dynamic-broker1#84
2017-09-08 13:58:13,213 | INFO | dynamic-broker1 bridge to dynamic-broker3
stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[dynamic-broker1] Task-161025
2017-09-08 13:58:14,194 | INFO | Establishing network connection from
vm://dynamic-broker1?async=false&network=true to tcp://192.168.1.138:61620 |
org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-12
2017-09-08 13:58:14,216 | WARN | Failed to add Connection
dynamic-broker1->dynamic-broker3-46135-1502950545007-1233:1 |
org.apache.activemq.broker.TransportConnection |
triggerStartAsyncNetworkBridgeCreation:
remoteBroker=tcp:///192.168.1.138:61620@50766, localBroker=
vm://dynamic-broker1#86
javax.jms.InvalidClientIDException: Broker: dynamic-broker1 - Client:
NC_dynamic-broker3_inbound_dynamic-broker1 already connected from
vm://dynamic-broker1#68
at
org.apache.activemq.broker.region.RegionBroker.addConnection(RegionBroker.java:255)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.jmx.ManagedRegionBroker.addConnection(ManagedRegionBroker.java:231)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:92)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:102)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:809)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:130)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:107)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startLocalBridge(DemandForwardingBridgeSupport.java:467)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.doStartLocalAndRemoteBridges(DemandForwardingBridgeSupport.java:420)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.access$500(DemandForwardingBridgeSupport.java:105)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$5.run(DemandForwardingBridgeSupport.java:335)[activemq-broker-5.11.1.jar:5.11.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2017-09-08 13:58:14,225 | INFO | Network connection between
vm://dynamic-broker1#86 and tcp:///192.168.1.138:61620@50766 shutdown due to a
local error: javax.jms.InvalidClientIDException: Broker: dynamic-broker1 -
Client: NC_dynamic-broker3_inbound_dynamic-broker1 already connected from
vm://dynamic-broker1#68 |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation:
remoteBroker=tcp:///192.168.1.138:61620@50766, localBroker=
vm://dynamic-broker1#86
2017-09-08 13:58:14,235 | INFO | dynamic-broker1 bridge to dynamic-broker3
stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[dynamic-broker1] Task-161016
2017-09-08 13:58:16,227 | INFO | Establishing network connection from
vm://dynamic-broker1?async=false&network=true to tcp://192.168.1.138:61620 |
org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-12
2017-09-08 13:58:16,252 | WARN | Failed to add Connection
dynamic-broker1->dynamic-broker3-46135-1502950545007-1235:1 |
org.apache.activemq.broker.TransportConnection |
triggerStartAsyncNetworkBridgeCreation:
remoteBroker=tcp:///192.168.1.138:61620@50767, localBroker=
vm://dynamic-broker1#88
javax.jms.InvalidClientIDException: Broker: dynamic-broker1 - Client:
NC_dynamic-broker3_inbound_dynamic-broker1 already connected from
vm://dynamic-broker1#68
at
org.apache.activemq.broker.region.RegionBroker.addConnection(RegionBroker.java:255)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.jmx.ManagedRegionBroker.addConnection(ManagedRegionBroker.java:231)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:92)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:102)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:809)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:130)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:107)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startLocalBridge(DemandForwardingBridgeSupport.java:467)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.doStartLocalAndRemoteBridges(DemandForwardingBridgeSupport.java:420)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport.access$500(DemandForwardingBridgeSupport.java:105)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$5.run(DemandForwardingBridgeSupport.java:335)[activemq-broker-5.11.1.jar:5.11.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
---------------------------------------------------------------------------
I find that Many others have similar mistakes,this one of them submitted a
issue:https://issues.apache.org/jira/browse/AMQ-6075
if this is a bug? what is the async error?and How can we solve this problem?
thanks very much
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
