[ https://issues.apache.org/jira/browse/ARTEMIS-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518479#comment-17518479 ]
Stephen Baker commented on ARTEMIS-3766: ---------------------------------------- In Scenario 2, this is the trace from A: {code:java} artemis-test-artemis-1-1 | 2022-04-06 22:06:54,508 FINE [proton.trace] IN: CH[0] : Transfer{handle=0, deliveryId=29, deliveryTag=\x01, messageFormat=0, settled=false, more=false, rcvSettleMode=null, state=null, resume=false, aborted=false, batchable=false}[\x00Sp\xc0\x02\x01A\x00Sr\xc1\x83\x08\xa3\x0fx-opt-amq-mr-qu\xa1\x02T3\xa3\x14x-opt-amq-mr-ev-type\xa3\x07postAck\xa3\x10x-opt-amq-mr-adr\xa1\x02T3\xa3\x10x-opt-amq-bkr-id\xa1$170c9c26-b5f5-11ec-ad18-0242ac140004\x00Sw\x81\x00\x00\x00\x00\x00\x00\x00\x95] artemis-test-artemis-1-1 | 2022-04-06 22:06:54,509 DEBUG [org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget] ActiveMQServerImpl::name=artemis-1 Post ack address=T3 queueName = T3 messageID=149, nodeID=170c9c26-b5f5-11ec-ad18-0242ac140004 artemis-test-artemis-1-1 | 2022-04-06 22:06:54,511 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueMessageMetrics] QueuePendingMessageMetrics[queue=T3, name=delivering] decrement messageCount to -2: Reference[149]:RELIABLE:CoreMessage[messageID=149,durable=true,userID=cdb68a7d-b5f5-11ec-8157-36d6a20c437d,priority=4, timestamp=Wed Apr 06 22:06:26 GMT 2022,expiration=Wed Apr 06 22:06:36 GMT 2022, durable=true, address=T3,size=220,properties=TypedProperties[__AMQ_CID=cda3c5ca-b5f5-11ec-8157-36d6a20c437d,_AMQ_ROUTING_TYPE=1]]@2028067055 artemis-test-artemis-1-1 | 2022-04-06 22:06:54,514 DEBUG [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843014: acknowledged message: Reference[149]:RELIABLE:CoreMessage[messageID=149,durable=true,userID=cdb68a7d-b5f5-11ec-8157-36d6a20c437d,priority=4, timestamp=Wed Apr 06 22:06:26 GMT 2022,expiration=Wed Apr 06 22:06:36 GMT 2022, durable=true, address=T3,size=220,properties=TypedProperties[__AMQ_CID=cda3c5ca-b5f5-11ec-8157-36d6a20c437d,_AMQ_ROUTING_TYPE=1]]@2028067055, with ackReason: NORMAL artemis-test-artemis-1-1 | 2022-04-06 22:06:54,515 INFO [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: acknowledged message ID: 149, messageRef sessionID: UNAVAILABLE, with messageRef consumerID: UNAVAILABLE, messageRef QueueName: T3, with ackReason: NORMAL {code} > Race condition with Dual Mirror and Expiry > ------------------------------------------ > > Key: ARTEMIS-3766 > URL: https://issues.apache.org/jira/browse/ARTEMIS-3766 > Project: ActiveMQ Artemis > Issue Type: Bug > Affects Versions: 2.21.0 > Reporter: Stephen Baker > Priority: Major > > When dual mirroring is in play there is a race condition which can result in > missing or duplicate messages on the brokers. > > Scenario 1: > # Bring up two artemis instances with broker-connection mirroring between > each other. (A and B) > # Send a message to A with a short TTL (I used 30 seconds in my test) > # After the message mirrors but before it expires, pause the Mirror queue on > both sides > # Wait for the message to expire, and the reaper thread to pick it up > # Observe that on both sides the message has moved to the ExpiryQueue. In > the Mirror queue there are 2 messages on A (a message destined to the > ExpiryQueue and an ack on the original message). On B there is a single > message in the mirror, which is the message to the expiry queue. > # Resume the mirror, I'm not sure the order matters, but I did B and then A. > # Observe that on A there is a single message in the ExpiryQueue, but on B > there are now 2 messages in the expiry queue. > Scenario 2: > # Bring up two artemis instances with broker-connection mirroring between > each other. (A and B). On A disable the reaper thread by setting > `<message-expiry-scan-period>-1</message-expiry-scan-period>` > # Send a message to A with a short TTL > # Wait for the TTL to expire > # On B the message moves to the ExpiryQueue, on A the message ends > disappears (no longer in the ExpiryQueue nor the original queue) -- This message was sent by Atlassian Jira (v8.20.1#820001)