[
https://issues.apache.org/jira/browse/ARTEMIS-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455252#comment-17455252
]
Tobias Månsson commented on ARTEMIS-3602:
-----------------------------------------
Here are logs from a failed delivery
December 8th 2021, 13:47:59.234 artemis-r4xg7 2021-12-08 12:47:59,234 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics]
QueuePendingMessageMetrics[queue=$.artemis.internal.sf.artemis.994daab0-51e4-11ec-bfd9-9ab74d93b535,
name=pending] increment messageCount to 825:
Reference[180276]:NON-RELIABLE:AMQPStandardMessage( [durable=false,
messageID=180276, address=test.amqp.websocket.in, size=583,
applicationProperties=\{msgType=test},
messageAnnotations=\{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180275},
properties=Properties\{messageId=null, userId=null,
to='test.amqp.websocket.in', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=Wed Dec 08 12:48:09 UTC 2021, creationTime=null,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ROUTE_TO$.artemis.internal.sf.artemis.994daab0-51e4-11ec-bfd9-9ab74d93b535=[0000
0000 0000
0087),bytesAsLongs(135],_AMQ_ORIG_ADDRESS=test.amqp.websocket.out,_AMQ_ORIG_MESSAGE_ID=180275,_AMQ_ORIG_QUEUE=test-amqp-websocket-1,_AMQ_AD=test.amqp.websocket.in]]
December 8th 2021, 13:47:59.234 artemis-r4xg7 2021-12-08 12:47:59,234 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue
$.artemis.internal.sf.artemis.994daab0-51e4-11ec-bfd9-9ab74d93b535 doing
deliver. messageReferences=824 with consumers=0
December 8th 2021, 13:47:59.233 artemis-r4xg7 2021-12-08 12:47:59,233 INFO
[com.husqvarnagroup.dss.husqiot.broker.interceptor.MessageInterceptor] INCOMING
interceptor accepted message with
MessageAnnotations\{{x-opt-websocketConnections=hib:husqtest:KB9rqflVDoECJ0Q=}}
and ApplicationProperties\{{msgType=test}} at test.amqp.websocket.out in 0 ms
December 8th 2021, 13:47:59.193 artemis-r4xg7 2021-12-08 12:47:59,192 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics]
QueuePendingMessageMetrics[queue=test.amqp.websocket.in, name=delivering]
decrement messageCount to 0:
Reference[180273]:NON-RELIABLE:AMQPStandardMessage( [durable=false,
messageID=180273, address=test.amqp.websocket.in, size=583,
applicationProperties=\{msgType=test},
messageAnnotations=\{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180272},
properties=Properties\{messageId=null, userId=null,
to='test.amqp.websocket.in', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=Wed Dec 08 12:48:09 UTC 2021, creationTime=null,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ORIG_ADDRESS=test.amqp.websocket.out,_AMQ_ORIG_MESSAGE_ID=180272,_AMQ_ORIG_QUEUE=test-amqp-websocket-1,_AMQ_AD=test.amqp.websocket.in]]
December 8th 2021, 13:47:59.190 artemis-r4xg7 2021-12-08 12:47:59,190 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics]
QueuePendingMessageMetrics[queue=test.amqp.websocket.in, name=pending]
decrement messageCount to 0:
Reference[180273]:NON-RELIABLE:AMQPStandardMessage( [durable=false,
messageID=180273, address=test.amqp.websocket.in, size=583,
applicationProperties=\{msgType=test},
messageAnnotations=\{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180272},
properties=Properties\{messageId=null, userId=null,
to='test.amqp.websocket.in', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=Wed Dec 08 12:48:09 UTC 2021, creationTime=null,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ORIG_ADDRESS=test.amqp.websocket.out,_AMQ_ORIG_MESSAGE_ID=180272,_AMQ_ORIG_QUEUE=test-amqp-websocket-1,_AMQ_AD=test.amqp.websocket.in]]
December 8th 2021, 13:47:59.190 artemis-r4xg7 2021-12-08 12:47:59,190 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics]
QueuePendingMessageMetrics[queue=test.amqp.websocket.in, name=delivering]
increment messageCount to 1:
Reference[180273]:NON-RELIABLE:AMQPStandardMessage( [durable=false,
messageID=180273, address=test.amqp.websocket.in, size=583,
applicationProperties=\{msgType=test},
messageAnnotations=\{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180272},
properties=Properties\{messageId=null, userId=null,
to='test.amqp.websocket.in', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=Wed Dec 08 12:48:09 UTC 2021, creationTime=null,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ORIG_ADDRESS=test.amqp.websocket.out,_AMQ_ORIG_MESSAGE_ID=180272,_AMQ_ORIG_QUEUE=test-amqp-websocket-1,_AMQ_AD=test.amqp.websocket.in]]
December 8th 2021, 13:47:59.190 artemis-r4xg7 2021-12-08 12:47:59,190 INFO
[com.husqvarnagroup.dss.husqiot.broker.interceptor.MessageInterceptor] OUTGOING
interceptor accepted message with
MessageAnnotations\{{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180272}} and
ApplicationProperties\{{msgType=test}} at test.amqp.websocket.in in 0 ms
December 8th 2021, 13:47:59.189 artemis-r4xg7 2021-12-08 12:47:59,189 INFO
[com.husqvarnagroup.dss.husqiot.broker.interceptor.MessageInterceptor] INCOMING
interceptor accepted message with
MessageAnnotations\{{x-opt-websocketConnections=hib:husqtest:KB9rqflVDoECJ0Q=}}
and ApplicationProperties\{{msgType=test}} at test.amqp.websocket.out in 1 ms
December 8th 2021, 13:47:59.189 artemis-r4xg7 2021-12-08 12:47:59,189 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue
test.amqp.websocket.in doing deliver. messageReferences=0 with consumers=1
December 8th 2021, 13:47:59.189 artemis-r4xg7 2021-12-08 12:47:59,189 DEBUG
[org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics]
QueuePendingMessageMetrics[queue=test.amqp.websocket.in, name=pending]
increment messageCount to 1:
Reference[180273]:NON-RELIABLE:AMQPStandardMessage( [durable=false,
messageID=180273, address=test.amqp.websocket.in, size=583,
applicationProperties=\{msgType=test},
messageAnnotations=\{x-opt-ORIG-QUEUE=test-amqp-websocket-1,
x-opt-ORIG-ADDRESS=test.amqp.websocket.out, x-opt-ORIG-MESSAGE-ID=180272},
properties=Properties\{messageId=null, userId=null,
to='test.amqp.websocket.in', subject='null', replyTo='null',
correlationId=null, contentType=null, contentEncoding=null,
absoluteExpiryTime=Wed Dec 08 12:48:09 UTC 2021, creationTime=null,
groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ORIG_ADDRESS=test.amqp.websocket.out,_AMQ_ORIG_MESSAGE_ID=180272,_AMQ_ORIG_QUEUE=test-amqp-websocket-1,_AMQ_AD=test.amqp.websocket.in]]
> Cluster connection not distribution messages
> --------------------------------------------
>
> Key: ARTEMIS-3602
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3602
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.18.0
> Reporter: Tobias Månsson
> Priority: Major
>
> The cluster is set up as a ON_DEMAND cluster with two nodes, were only one
> node is exposed to the external load-balancer and receiving new connections.
> This creates a online failover setup by switching over new connections to the
> other node, before taking down the current for maintenance.
> During one deployment, the active broker node was failed over without
> shutting down the passive node. This left the current connection on the
> passive node, while new connections were created against the now active one.
> If message were sent to the now passive node, they were synchronized
> correctly to both nodes and to all connections. But if the messages were sent
> to then now active node (the intended behavior), they were not sent to
> clients currently connected to the now passive node.
> The logs and message counters confirm this, but no cause has been found.
> Since the cluster synchronization work one way, but not the other, I don't
> know where to search for errors. I've enabled debug logs on both servers, but
> don't see any issues. The messages are committed to the correct queues, but
> nothing is received on the other node in the failure scenario.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
