[
https://issues.apache.org/jira/browse/ARTEMIS-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy A. Bish closed ARTEMIS-2879.
------------------------------------
Resolution: Cannot Reproduce
Without a reproducer this can't be validated, the linked issue appears to
handle a similar problem that has been addressed.
> Artemis unable to reconnect to client after client restart
> ----------------------------------------------------------
>
> Key: ARTEMIS-2879
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2879
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.14.0
> Reporter: Vesa Koskela
> Priority: Major
>
> We recently attempted to upgrade our Artemis from version 2.11.0 to 2.14.0 in
> our development environment. Our current CI/CD pipeline installs the nightly
> build to this environment, which leads to all Artemis clients being updated
> and restarted. We noticed that after the restart the client is unable to
> connect to Artemis.
> Below are some snippets of what we found from Artemis logs.
> Right after the nightly build was installed:
> {code}
> 2020-08-12 02:03:45,728 WARN [org.apache.activemq.artemis.core.server]
> AMQ222061: Client connection failed, clearing up resources for session
> 07ba2a53-dc40-11ea-827f-0021f6976e26
>
> 2020-08-12 02:03:45,800 WARN
> [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for
> session 07ba2a53-dc40-11ea-827f-0021f6976e26
>
> 2020-08-12 02:03:45,800
> WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection
> failed, clearing up resources for session
> 07ba2a54-dc40-11ea-827f-0021f6976e26
>
> 2020-08-12 02:03:45,801 WARN
> [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for
> session 07ba2a54-dc40-11ea-827f-0021f6976e26
>
> 2020-08-12 02:03:45,866
> WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection
> failed, clearing up resources for session
> 28d62691-db93-11ea-bd51-0021f6976e26
>
> 2020-08-12 02:03:45,866 WARN
> [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for
> session 28d62691-db93-11ea-bd51-0021f6976e26
>
> 2020-08-12 02:03:45,867
> WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection
> failed, clearing up resources for session
> 28d64da2-db93-11ea-bd51-0021f6976e26
>
> 2020-08-12 02:03:45,867 WARN
> [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for
> session 28d64da2-db93-11ea-bd51-0021f6976e26
>
> 2020-08-12 02:03:45,867
> WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection
> failed, clearing up resources for session
> 2eaa760c-db93-11ea-bd51-0021f6976e26
>
>
>
>
> {code}
> This continues for 178 lines. Then we get an exception:
> {code}
> 2020-08-12 02:03:46,138 ERROR
> [org.apache.activemq.artemis.core.journal.impl.JournalImpl]
> appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id
> 8233509: java.lang.IllegalStateException: Cannot find tx with id 8233509
> at
> org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451)
> [artemis-journal-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [java.base:]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [java.base:]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.14.0.jar:2.14.0]
> {code}
> Then we get a couple of {{WARN [org.apache.activemq.artemis.core.server]
> AMQ222104: IO Error completing the transaction, code = 6, message = Cannot
> find tx with id 8233509}} lines, followed up by two exceptions:
> {code}
> 2020-08-12 02:03:46,216 ERROR
> [org.apache.activemq.artemis.core.journal.impl.JournalImpl]
> appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id
> 8233390: java.lang.IllegalStateException: Cannot find tx with id 8233390
> at
> org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451)
> [artemis-journal-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [java.base:]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [java.base:]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.14.0.jar:2.14.0]
> 2020-08-12 02:03:46,220 ERROR
> [org.apache.activemq.artemis.core.journal.impl.JournalImpl]
> appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id
> 8232236: java.lang.IllegalStateException: Cannot find tx with id 8232236
> at
> org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451)
> [artemis-journal-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
> [artemis-commons-2.14.0.jar:2.14.0]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [java.base:]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [java.base:]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.14.0.jar:2.14.0]
> {code}
> After that we get some more failed client connections, resource cleaning and
> now also some failed reattach requests.
> {code}
> 2020-08-12 02:03:54,000 WARN [org.apache.activemq.artemis.core.server]
> AMQ222061: Client connection failed, clearing up resources for session
> 0fcaadee-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server]
> AMQ222104: IO Error completing the transaction, code = 6, message = Cannot
> find tx with id 8232236
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server]
> AMQ222107: Cleared up resources for session
> 0fcaadee-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server]
> AMQ222061: Client connection failed, clearing up resources for session
> 0fcafc0f-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server]
> AMQ222107: Cleared up resources for session
> 0fcafc0f-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,009 WARN [org.apache.activemq.artemis.core.server]
> AMQ222066: Reattach request from /<server-ip>:45829 failed as there is no
> confirmationWindowSize configured, which may be ok for your system
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server]
> AMQ222061: Client connection failed, clearing up resources for session
> 12d2bde3-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server]
> AMQ222107: Cleared up resources for session
> 12d2bde3-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server]
> AMQ222061: Client connection failed, clearing up resources for session
> 12e31194-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server]
> AMQ222104: IO Error completing the transaction, code = 6, message = Cannot
> find tx with id 8232236
> 2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server]
> AMQ222107: Cleared up resources for session
> 12e31194-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server]
> AMQ222066: Reattach request from /<server-ip>:46129 failed as there is no
> confirmationWindowSize configured, which may be ok for your system
> {code}
> These resource clearings, failed reattach requests, client connection
> failures and occasional exceptions continue until Artemis is restarted. After
> that Artemis works until the next nightly build is installed.
> We rolled back to 2.11.0 and everything works again like it should be. So the
> problem seems to be mainly on how 2.14.0 handles reconnections after client
> is reinstalled.
> Most of our client applications are using artemis client version 2.10.1,
> including the ones that are having these problems. This problem manifests
> with only a couple of applications though, which is why I don't think the
> client version incompatibility is a problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact
