Hi,
We have 3 node symmetric cluster, with server side load balancing
(ON_DEMAND) and message redistribution enabled.
When using <address-full-policy>FAIL<address-full-policy>, we noticed that
amqp producers got an exception as expected, however the bridge connection
also got the exception and disconnected with below error. This affected
load balancing/redistribution for all addresses.
Is this expected behaviour or some setting at bridge level which can fix
this problem?
2023-06-01 05:58:04,496 WARN [org.apache.activemq.artemis.core.server]
AMQ222703: Address "..." is full. Bridge
$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96
will disconnect
2023-06-01 05:58:04,497 WARN [org.apache.activemq.artemis.core.server]
AMQ222151: removing consumer which did not handle a message,
consumer=ClusterConnectionBridge@1bb7d712
[name=$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96,
queue=QueueImpl[name=$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96,
postOffice=PostOfficeImpl
[server=ActiveMQServerImpl::name=activemq-artemis-2], temp=false]@33cbd3f2
targetConnector=ServerLocatorImpl
(identity=(Cluster-connection-bridge::ClusterConnectionBridge@1bb7d712
[name=$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96,
queue=QueueImpl[name=$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96,
postOffice=PostOfficeImpl
[server=ActiveMQServerImpl::name=activemq-artemis-2], temp=false]@33cbd3f2
targetConnector=ServerLocatorImpl
[initialConnectors=[TransportConfiguration(name=artemis-connector,
factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory)
?trustStorePassword=****&forceSSLParameters=false&port=61616&keyStorePassword=****&sslEnabled=true&host=activemq-artemis-1-activemq-artemis-activemq-artemis-svc-cluster-local&verifyHost=false&trustStorePath=server-truststore-ks&keyStorePath=server-keystore-ks],
discoveryGroupConfiguration=null]]::ClusterConnectionImpl@1971047138[nodeUUID=f2367795-9b65-11eb-86fe-f26214a34b40,
connector=TransportConfiguration(name=artemis-connector,
factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory)
?trustStorePassword=****&forceSSLParameters=false&port=61616&keyStorePassword=****&sslEnabled=true&host=activemq-artemis-2-activemq-artemis-activemq-artemis-svc-cluster-local&verifyHost=false&trustStorePath=server-truststore-ks&keyStorePath=server-keystore-ks,
address=, server=ActiveMQServerImpl::name=activemq-artemis-2]))
[initialConnectors=[TransportConfiguration(name=artemis-connector,
factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory)
?trustStorePassword=****&forceSSLParameters=false&port=61616&keyStorePassword=****&sslEnabled=true&host=activemq-artemis-1-activemq-artemis-activemq-artemis-svc-cluster-local&verifyHost=false&trustStorePath=server-truststore-ks&keyStorePath=server-keystore-ks],
discoveryGroupConfiguration=null]],
message=Reference[51332165303]:RELIABLE:AMQPStandardMessage( [durable=true,
messageID=51332165303, address=vcc.rvdc.results, size=45006,
scanningStatus=SCANNED, applicationProperties={},
messageAnnotations={x-opt-jms-dest=0, x-opt-jms-msg-type=5},
properties=Properties{messageId=ID:04b5c8fd-fea9-4ef1-8aea-af83d23cbca3:111:3:2-742584,
userId=null, to='vcc.rvdc.results', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=null, creationTime=Thu Jun 01 05:58:04 UTC 2023,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties
=
TypedProperties[_AMQ_ROUTE_TO$.artemis.internal.sf.activemq-artemis.5ef93963-d7e7-11ea-bf87-9add4963fe96=[0000
0001 3722 3C33),bytesAsLongs(5219957811]]]
java.lang.NullPointerException: Cannot invoke
"org.apache.activemq.artemis.core.client.impl.ClientSessionInternal.isWritable(org.apache.activemq.artemis.spi.core.remoting.ReadyListener)"
because "this.session" is null
at
org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl.handle(BridgeImpl.java:558)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.handle(QueueImpl.java:3980)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3127)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4298)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
~[artemis-commons-2.28.0.jar:?]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
~[artemis-commons-2.28.0.jar:?]
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
~[artemis-commons-2.28.0.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
~[?:?]
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
~[artemis-commons-2.28.0.jar:?]
2023-06-01 05:58:04,508 WARN
[org.apache.activemq.artemis.core.server.impl.QueueImpl] null
java.util.NoSuchElementException: null
at
org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl$PriorityLinkedListIterator.repeat(PriorityLinkedListImpl.java:225)
~[artemis-commons-2.28.0.jar:?]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3150)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4298)
~[artemis-server-2.28.0.jar:2.28.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
~[artemis-commons-2.28.0.jar:?]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
~[artemis-commons-2.28.0.jar:?]
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
~[artemis-commons-2.28.0.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
~[?:?]
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
~[artemis-commons-2.28.0.jar:?]
Cluster configuration
<cluster-connections>
<cluster-connection name="activemq-artemis">
<address></address>
<connector-ref>artemis-connector</connector-ref>
<check-period>1000</check-period>
<connection-ttl>5000</connection-ttl>
<min-large-message-size>500000</min-large-message-size>
<call-timeout>5000</call-timeout>
<retry-interval>500</retry-interval>
<retry-interval-multiplier>0.5</retry-interval-multiplier>
<max-retry-interval>40000</max-retry-interval>
<initial-connect-attempts>-1</initial-connect-attempts>
<reconnect-attempts>-1</reconnect-attempts>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<confirmation-window-size>104857600</confirmation-window-size>
<call-failover-timeout>30000</call-failover-timeout>
<notification-interval>1000</notification-interval>
<notification-attempts>2</notification-attempts>
<discovery-group-ref
discovery-group-name="artemis-discovery-group"/>
</cluster-connection>
</cluster-connections>
Best Regards
Tapan Gupta
