Your patch does resolve the error. Artemis 2.27 looks like it will be a difficult upgrade, I ended up making a new instance and merging config over.
I have pasted a trace in https://issues.apache.org/jira/browse/ARTEMIS-4045 What is the impact of this issue? I’m trying to decide whether to advise our IT team to continue with the planned upgrade or hold off until 2.27. We will definitely encounter this condition in production. From a surface reading, possibly a resource leak? From: Clebert Suconic <clebert.suco...@gmail.com> Date: Wednesday, October 12, 2022 at 9:54 PM To: users@activemq.apache.org <users@activemq.apache.org> Subject: Re: Mirror compatibility across versions Notice that main is now using SLF4j / log4j... (in case you manually upgrade to a snapshot) We are still working the details for an upgrade. if you need to patch your 2.25.0 it's a straight change to make there On Wed, Oct 12, 2022 at 9:52 PM Clebert Suconic <clebert.suco...@gmail.com> wrote: > > I don't know how I would test it yet. It's fairly late in the night > for me.. I will think about it tomorrow. > > > but here is a tentative fix: > > https://github.com/apache/activemq-artemis/pull/4256 > > On Wed, Oct 12, 2022 at 9:30 PM Stephen Baker > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > That’s the full output with regular logging levels. I can reproduce at will > > so I have enabled trace level logging and pasted the result in > > https://issues.apache.org/jira/browse/ARTEMIS-4045 > > > > Let’s take further discussion there? > > > > From: Clebert Suconic <clebert.suco...@gmail.com> > > Date: Wednesday, October 12, 2022 at 9:10 PM > > To: users@activemq.apache.org <users@activemq.apache.org> > > Subject: Re: Mirror compatibility across versions > > is this the actual trace? or you cut some to post here? > > > > > > Just puzzled by skipDelivery calling performAck.. > > > > artemis-test-artemis-1-m-1 | at > > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.lambda$performAck$2(AMQPMirrorControllerTarget.java:377) > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > artemis-test-artemis-1-m-1 | at > > org.apache.activemq.artemis.core.server.impl.QueueImpl$2.skipDelivery(QueueImpl.java:1203) > > [artemis-server-2.25.0.jar:2.25.0] > > > > > > can you post the full stack if this is not it? > > > > > > it definitely needs fixing... I'm investigating it. > > > > On Wed, Oct 12, 2022 at 6:05 PM Stephen Baker > > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > > > Having updated both sides to 2.25 I’m seeing this error in the logs, is > > > it a concern that warrants further investigation? > > > > > > artemis-test-artemis-1-m-1 | 2022-10-12 22:01:43,632 ERROR > > > [org.apache.activemq.artemis.core.server] AMQ224041: Failed to deliver: > > > java.lang.IllegalStateException: this method requires to be called within > > > the handler, use the executor > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.requireHandler(ProtonHandler.java:210) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.requireInHandler(AMQPConnectionContext.java:197) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.proton.ProtonAbstractReceiver.settle(ProtonAbstractReceiver.java:185) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget$ACKMessageOperation.run(AMQPMirrorControllerTarget.java:125) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.performAck(AMQPMirrorControllerTarget.java:388) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.lambda$performAck$2(AMQPMirrorControllerTarget.java:377) > > > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.core.server.impl.QueueImpl$2.skipDelivery(QueueImpl.java:1203) > > > [artemis-server-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.core.server.impl.QueueImpl.doInternalPoll(QueueImpl.java:2932) > > > [artemis-server-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2991) > > > [artemis-server-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4250) > > > [artemis-server-2.25.0.jar:2.25.0] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:56) > > > [artemis-commons-2.25.0.jar:] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) > > > [artemis-commons-2.25.0.jar:] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:67) > > > [artemis-commons-2.25.0.jar:] > > > artemis-test-artemis-1-m-1 | at > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > [java.base:] > > > artemis-test-artemis-1-m-1 | at > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > [java.base:] > > > artemis-test-artemis-1-m-1 | at > > > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) > > > [artemis-commons-2.25.0.jar:] > > > > > > > > > From: Stephen Baker <stephen.ba...@rmssoftwareinc.com> > > > Date: Wednesday, October 12, 2022 at 4:43 PM > > > To: users@activemq.apache.org <users@activemq.apache.org> > > > Subject: Re: Mirror compatibility across versions > > > I set up some docker images in this configuration as a preliminary step. > > > One oddity: > > > > > > Configure 2.25 side not to run the reaper > > > Send message to 2.25 side > > > Observe that after expiry the message shows up in the expiry queue on the > > > 2.20 side, but not on the 2.25 side, the message is removed from the > > > original queue on both sides. > > > > > > If the message is originally sent to the 2.20 side it shows up in both > > > queues as expected. > > > > > > There’s probably a reason for it, but I didn’t expect this change. I > > > thought that we would continue to see the old bugs until both sides were > > > updated. > > > > > > From: Clebert Suconic <clebert.suco...@gmail.com> > > > Date: Tuesday, October 11, 2022 at 3:24 PM > > > To: users@activemq.apache.org <users@activemq.apache.org> > > > Subject: Re: Mirror compatibility across versions > > > Yeah.. something like that... not necessarily in there though. but a > > > similar test. > > > > > > On Tue, Oct 11, 2022 at 1:44 PM Stephen Baker > > > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > > > > > Ok, I agree based on a cursory reading of that patch. The extra > > > > ackReason defaults to normal in one direction and isn’t read in the > > > > other direction. Killed, replaced, and expired being interpreted as > > > > normal just means that the 2.20 bugs will persist until both sides are > > > > updated. > > > > > > > > I’ll test it out with different version docker containers. I suppose as > > > > far as writing tests you mean something like the > > > > MultiVersionReplicaTest. > > > > > > > > Stephen E. Baker > > > > > > > > > > > > From: Clebert Suconic <clebert.suco...@gmail.com> > > > > Date: Tuesday, October 11, 2022 at 12:59 PM > > > > To: users@activemq.apache.org <users@activemq.apache.org> > > > > Subject: Re: Mirror compatibility across versions > > > > In theory it should work. > > > > > > > > > > > > Only change that might break compatibility is > > > > https://github.com/apache/activemq-artemis/commit/68f6d8263d8c795722805f0e4d6939e7a8b9ed48 > > > > which is ARTEMIS-3743 / ARTEMIS-3766 Use ACKReason on Mirror to > > > > determine target operations and fixing Delivering statistics on Mirror > > > > > > > > > > > > I tried to not break compatibility, but I just realized we should add > > > > a test to validate compatibility between mirrors. > > > > > > > > > > > > > > > > so, I will say it should be compatible, but I would test it before > > > > doing it in the real system. > > > > > > > > > > > > > > > > if you are willing to contribute to a compatibility test :) > > > > > > > > On Tue, Oct 11, 2022 at 10:06 AM Stephen Baker > > > > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > > > > > > > We are planning our production upgrade from 2.20 to 2.25. These > > > > > upgrades involve a loss of service in the window between stopping the > > > > > live and when the backup instance becomes ready to process messages. > > > > > > > > > > I was wondering if the mirror protocol is expected to be compatible > > > > > between those versions. If so we could upgrade our cold site, and > > > > > then wait for a planned failover to avoid any additional down time. I > > > > > know that quite a bit of work was done by Clebert in 2.24 so I was > > > > > hoping he could weigh in. > > > > > > > > > > Stephen E Baker > > > > > > > > > > > > > > > > -- > > > > Clebert Suconic > > > > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. > > > > Do not click links or open attachments unless you recognize the sender > > > > and know the content is safe. > > > > > > > > > > > > -- > > > Clebert Suconic > > > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do > > > not click links or open attachments unless you recognize the sender and > > > know the content is safe. > > > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do > > > not click links or open attachments unless you recognize the sender and > > > know the content is safe. > > > > > > > > -- > > Clebert Suconic > > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do > > not click links or open attachments unless you recognize the sender and > > know the content is safe. > > > > -- > Clebert Suconic -- Clebert Suconic [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do not click links or open attachments unless you recognize the sender and know the content is safe.
