Hi Team,
I have requirement to create Artemis master slave shared storage cluster using Postgres database in Kubernetes. Here is *Master broker.xml* <?xml version='1.0'?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration xmlns="urn:activemq" xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xi=http://www.w3.org/2001/XInclude xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd"> <core xmlns="urn:activemq:core" xmlns:xsi= http://www.w3.org/2001/XMLSchema-instance xsi:schemaLocation="urn:activemq:core "> <name>queue-a</name> <store> <database-store> <!-- The most efficient persistent layer for Artemis is the file-store, however if you require a database please refer to your database provider for any database specific questions. We don't endorse any specific JDBC provider. Derby is provided by default for demonstration purposes. --> <jdbc-driver-class-name>org.postgresql.Driver</jdbc-driver-class-name> <jdbc-connection-url>Database Connection URL</jdbc-connection-url> <message-table-name>MESSAGES</message-table-name> <bindings-table-name>BINDINGS</bindings-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-lock-renew-period>4000</jdbc-lock-renew-period> <jdbc-network-timeout>20000</jdbc-network-timeout> </database-store> </store> <persistence-enabled>true</persistence-enabled> <!-- this could be ASYNCIO, MAPPED, NIO ASYNCIO: Linux Libaio MAPPED: mmap files NIO: Plain Java Files --> <journal-type>NIO</journal-type> <paging-directory>data/paging</paging-directory> <bindings-directory>data/bindings</bindings-directory> <journal-directory>data/journal</journal-directory> <large-messages-directory>data/large-messages</large-messages-directory> <!-- if you want to retain your journal uncomment this following configuration. This will allow your system to keep 7 days of your data, up to 10G. Tweak it accordingly to your use case and capacity. it is recommended to use a separate storage unit from the journal for performance considerations. <journal-retention-directory period="7" unit="DAYS" storage-limit="10G">data/retention</journal-retention-directory> You can also enable retention by using the argument journal-retention on the `artemis create` command --> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>10</journal-pool-files> <journal-device-block-size>4096</journal-device-block-size> <journal-file-size>10M</journal-file-size> <!-- You can verify the network health of a particular NIC by specifying the <network-check-NIC> element. <network-check-NIC>theNicName</network-check-NIC> --> <!-- Use this to use an HTTP server to validate the network <network-check-URL-list> http://www.apache.org</network-check-URL-list> --> <!-- <network-check-period>10000</network-check-period> --> <!-- <network-check-timeout>1000</network-check-timeout> --> <!-- this is a comma separated list, no spaces, just DNS or IPs it should accept IPV6 Warning: Make sure you understand your network topology as this is meant to validate if your network is valid. Using IPs that could eventually disappear or be partially visible may defeat the purpose. You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running --> <!-- <network-check-list>10.0.0.1</network-check-list> --> <!-- use this to customize the ping used for ipv4 addresses --> <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> --> <!-- use this to customize the ping used for ipv6 addresses --> <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> --> <connectors> <!-- Connector used to be announced through cluster connections and notifications --> <connector name="artemis">tcp://queue-a:61616</connector> </connectors> <!-- how often we are looking for how many bytes are being used on the disk in ms --> <disk-scan-period>5000</disk-scan-period> <!-- once the disk hits this limit the system will block, or close the connection in certain protocols that won't support flow control. --> <max-disk-usage>90</max-disk-usage> <!-- should the broker detect dead locks and other issues --> <critical-analyzer>true</critical-analyzer> <critical-analyzer-timeout>120000</critical-analyzer-timeout> <critical-analyzer-check-period>60000</critical-analyzer-check-period> <critical-analyzer-policy>HALT</critical-analyzer-policy> <!-- the system will enter into page mode once you hit this limit. This is an estimate in bytes of how much the messages are using in memory The system will use half of the available memory (-Xmx) by default for the global-max-size. You may specify a different value here if you need to customize it to your needs. <global-max-size>100Mb</global-max-size> --> <acceptors> <!-- useEpoll means: it will use Netty epoll if you are on a system (Linux) that supports it --> <!-- amqpCredits: The number of credits sent to AMQP producers --> <!-- amqpLowCredits: The server will send the # credits specified at amqpCredits at this low mark --> <!-- amqpDuplicateDetection: If you are not using duplicate detection, set this to false as duplicate detection requires applicationProperties to be parsed on the server. --> <!-- amqpMinLargeMessageSize: Determines how many bytes are considered large, so we start using files to hold their data. default: 102400, -1 would mean to disable large mesasge control --> <!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add "anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url. See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. --> <!-- Acceptor for every supported protocol --> <acceptor name="artemis">tcp://queue-a:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNET Q,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=false;suppressInternalManagementObjects=false</acceptor> <!-- AMQP Acceptor. Listens on default AMQP port for AMQP traffic.--> <acceptor name="amqp">tcp://queue-a:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;am qpMinLargeMessageSize=102400;amqpDuplicateDetection=true</acceptor> <!-- STOMP Acceptor. --> <acceptor name="stomp">tcp://queue-a:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor> <!-- HornetQ Compatibility Acceptor. Enables HornetQ Core and STOMP for legacy HornetQ clients. --> <acceptor name="hornetq">tcp://queue-a:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor> <!-- MQTT Acceptor --> <acceptor name="mqtt">tcp://queue-a:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor> </acceptors> <cluster-user>admin</cluster-user> <cluster-password>admin</cluster-password> <broadcast-groups> <broadcast-group name="bg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <broadcast-period>5000</broadcast-period> <connector-ref>artemis</connector-ref> </broadcast-group> </broadcast-groups> <discovery-groups> <discovery-group name="dg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <refresh-timeout>10000</refresh-timeout> </discovery-group> </discovery-groups> <cluster-connections> <cluster-connection name="my-cluster"> <connector-ref>artemis</connector-ref> <message-load-balancing>ON_DEMAND</message-load-balancing> <max-hops>0</max-hops> <discovery-group-ref discovery-group-name="dg-group1"/> </cluster-connection> </cluster-connections> <ha-policy> <shared-store> <master> <failover-on-shutdown>true</failover-on-shutdown> </master> </shared-store> </ha-policy> <security-settings> <security-setting match="#"> <permission type="createNonDurableQueue" roles="amq"/> <permission type="deleteNonDurableQueue" roles="amq"/> <permission type="createDurableQueue" roles="amq"/> <permission type="deleteDurableQueue" roles="amq"/> <permission type="createAddress" roles="amq"/> <permission type="deleteAddress" roles="amq"/> <permission type="consume" roles="amq"/> <permission type="browse" roles="amq"/> <permission type="send" roles="amq"/> <!-- we need this otherwise ./artemis data imp wouldn't work --> <permission type="manage" roles="amq"/> </security-setting> </security-settings> <address-settings> <!-- if you define auto-create on certain queues, management has to be auto-create --> <address-setting match="activemq.management#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <!--default for catch all--> <address-setting match="#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> <auto-delete-queues>false</auto-delete-queues> <auto-delete-addresses>false</auto-delete-addresses> </address-setting> </address-settings> <addresses> <address name="DLQ"> <anycast> <queue name="DLQ" /> </anycast> </address> <address name="ExpiryQueue"> <anycast> <queue name="ExpiryQueue" /> </anycast> </address> </addresses> <!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events <broker-plugins> <broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin"> <property key="LOG_ALL_EVENTS" value="true"/> <property key="LOG_CONNECTION_EVENTS" value="true"/> <property key="LOG_SESSION_EVENTS" value="true"/> <property key="LOG_CONSUMER_EVENTS" value="true"/> <property key="LOG_DELIVERING_EVENTS" value="true"/> <property key="LOG_SENDING_EVENTS" value="true"/> <property key="LOG_INTERNAL_EVENTS" value="true"/> </broker-plugin> </broker-plugins> --> </core> </configuration> *Slave broker.xml* <?xml version='1.0'?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration xmlns="urn:activemq" xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xi=http://www.w3.org/2001/XInclude xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd"> <core xmlns="urn:activemq:core" xmlns:xsi= http://www.w3.org/2001/XMLSchema-instance xsi:schemaLocation="urn:activemq:core "> <name>queue-b</name> <store> <database-store> <!-- The most efficient persistent layer for Artemis is the file-store, however if you require a database please refer to your database provider for any database specific questions. We don't endorse any specific JDBC provider. Derby is provided by default for demonstration purposes. --> <jdbc-driver-class-name>org.postgresql.Driver</jdbc-driver-class-name> <jdbc-connection-url>jdbc:postgresql:// pxinfra-development.cluster-cvresftoa15j.us-east-2.rds.amazonaws.com:5432/postgres-platformx-pe-429-artemis?user=postgres &password=Provenir1</jdbc-connection-url> <message-table-name>MESSAGES</message-table-name> <bindings-table-name>BINDINGS</bindings-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-lock-renew-period>4000</jdbc-lock-renew-period> <jdbc-network-timeout>20000</jdbc-network-timeout> </database-store> </store> <persistence-enabled>true</persistence-enabled> <!-- this could be ASYNCIO, MAPPED, NIO ASYNCIO: Linux Libaio MAPPED: mmap files NIO: Plain Java Files --> <journal-type>NIO</journal-type> <paging-directory>data/paging</paging-directory> <bindings-directory>data/bindings</bindings-directory> <journal-directory>data/journal</journal-directory> <large-messages-directory>data/large-messages</large-messages-directory> <!-- if you want to retain your journal uncomment this following configuration. This will allow your system to keep 7 days of your data, up to 10G. Tweak it accordingly to your use case and capacity. it is recommended to use a separate storage unit from the journal for performance considerations. <journal-retention-directory period="7" unit="DAYS" storage-limit="10G">data/retention</journal-retention-directory> You can also enable retention by using the argument journal-retention on the `artemis create` command --> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>10</journal-pool-files> <journal-device-block-size>4096</journal-device-block-size> <journal-file-size>10M</journal-file-size> <!-- You can verify the network health of a particular NIC by specifying the <network-check-NIC> element. <network-check-NIC>theNicName</network-check-NIC> --> <!-- Use this to use an HTTP server to validate the network <network-check-URL-list> http://www.apache.org</network-check-URL-list> --> <!-- <network-check-period>10000</network-check-period> --> <!-- <network-check-timeout>1000</network-check-timeout> --> <!-- this is a comma separated list, no spaces, just DNS or IPs it should accept IPV6 Warning: Make sure you understand your network topology as this is meant to validate if your network is valid. Using IPs that could eventually disappear or be partially visible may defeat the purpose. You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running --> <!-- <network-check-list>10.0.0.1</network-check-list> --> <!-- use this to customize the ping used for ipv4 addresses --> <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> --> <!-- use this to customize the ping used for ipv6 addresses --> <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> --> <connectors> <!-- Connector used to be announced through cluster connections and notifications --> <connector name="artemis">tcp://queue-b:61616</connector> </connectors> <!-- how often we are looking for how many bytes are being used on the disk in ms --> <disk-scan-period>5000</disk-scan-period> <!-- once the disk hits this limit the system will block, or close the connection in certain protocols that won't support flow control. --> <max-disk-usage>90</max-disk-usage> <!-- should the broker detect dead locks and other issues --> <critical-analyzer>true</critical-analyzer> <critical-analyzer-timeout>120000</critical-analyzer-timeout> <critical-analyzer-check-period>60000</critical-analyzer-check-period> <critical-analyzer-policy>HALT</critical-analyzer-policy> <!-- the system will enter into page mode once you hit this limit. This is an estimate in bytes of how much the messages are using in memory The system will use half of the available memory (-Xmx) by default for the global-max-size. You may specify a different value here if you need to customize it to your needs. <global-max-size>100Mb</global-max-size> --> <acceptors> <!-- useEpoll means: it will use Netty epoll if you are on a system (Linux) that supports it --> <!-- amqpCredits: The number of credits sent to AMQP producers --> <!-- amqpLowCredits: The server will send the # credits specified at amqpCredits at this low mark --> <!-- amqpDuplicateDetection: If you are not using duplicate detection, set this to false as duplicate detection requires applicationProperties to be parsed on the server. --> <!-- amqpMinLargeMessageSize: Determines how many bytes are considered large, so we start using files to hold their data. default: 102400, -1 would mean to disable large mesasge control --> <!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add "anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url. See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. --> <!-- Acceptor for every supported protocol --> <acceptor name="artemis">tcp://queue-b:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNET Q,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;supportAdvisory=false;suppressInternalManagementObjects=false</acceptor> <!-- AMQP Acceptor. Listens on default AMQP port for AMQP traffic.--> <acceptor name="amqp">tcp://queue-b:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;am qpMinLargeMessageSize=102400;amqpDuplicateDetection=true</acceptor> <!-- STOMP Acceptor. --> <acceptor name="stomp">tcp://queue-b:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor> <!-- HornetQ Compatibility Acceptor. Enables HornetQ Core and STOMP for legacy HornetQ clients. --> <acceptor name="hornetq">tcp://queue-b:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor> <!-- MQTT Acceptor --> <acceptor name="mqtt">tcp://queue-b:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor> </acceptors> <cluster-user>admin</cluster-user> <cluster-password>admin</cluster-password> <broadcast-groups> <broadcast-group name="bg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <broadcast-period>5000</broadcast-period> <connector-ref>artemis</connector-ref> </broadcast-group> </broadcast-groups> <discovery-groups> <discovery-group name="dg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <refresh-timeout>10000</refresh-timeout> </discovery-group> </discovery-groups> <cluster-connections> <cluster-connection name="my-cluster"> <connector-ref>artemis</connector-ref> <message-load-balancing>ON_DEMAND</message-load-balancing> <max-hops>0</max-hops> <discovery-group-ref discovery-group-name="dg-group1"/> </cluster-connection> </cluster-connections> <ha-policy> <shared-store> <slave> <failover-on-shutdown>true</failover-on-shutdown> </slave> </shared-store> </ha-policy> <security-settings> <security-setting match="#"> <permission type="createNonDurableQueue" roles="amq"/> <permission type="deleteNonDurableQueue" roles="amq"/> <permission type="createDurableQueue" roles="amq"/> <permission type="deleteDurableQueue" roles="amq"/> <permission type="createAddress" roles="amq"/> <permission type="deleteAddress" roles="amq"/> <permission type="consume" roles="amq"/> <permission type="browse" roles="amq"/> <permission type="send" roles="amq"/> <!-- we need this otherwise ./artemis data imp wouldn't work --> <permission type="manage" roles="amq"/> </security-setting> </security-settings> <address-settings> <!-- if you define auto-create on certain queues, management has to be auto-create --> <address-setting match="activemq.management#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <!--default for catch all--> <address-setting match="#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> <auto-delete-queues>false</auto-delete-queues> <auto-delete-addresses>false</auto-delete-addresses> </address-setting> </address-settings> <addresses> <address name="DLQ"> <anycast> <queue name="DLQ" /> </anycast> </address> <address name="ExpiryQueue"> <anycast> <queue name="ExpiryQueue" /> </anycast> </address> </addresses> <!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events <broker-plugins> <broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin"> <property key="LOG_ALL_EVENTS" value="true"/> <property key="LOG_CONNECTION_EVENTS" value="true"/> <property key="LOG_SESSION_EVENTS" value="true"/> <property key="LOG_CONSUMER_EVENTS" value="true"/> <property key="LOG_DELIVERING_EVENTS" value="true"/> <property key="LOG_SENDING_EVENTS" value="true"/> <property key="LOG_INTERNAL_EVENTS" value="true"/> </broker-plugin> </broker-plugins> --> </core> </configuration> Master is able to start properly but when I start slave, it is giving below error 2023-07-24 12:23:58,529 DEBUG [org.apache.activemq.artemis.core.server.impl.jdbc.JdbcLeaseLock] [LIVE] ec83a09c-c9c8-42e6-8999-eaaf10831175 has failed to acquire lock: lock status = { holderId = 182db189-2588-4c85-b78b-72c28a5ca9a1 expirationTime = 2023-07-24 12:24:17.562 currentTimestamp = 2023-07-24 12:23:58.525833 } 2023-07-24 12:23:58,529 FINE [org.postgresql.jdbc.PgConnection] setAutoCommit = true 2023-07-24 12:24:00,185 DEBUG [org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl] ***************************************************************************** ** Complete Thread dump Discovery timeout, printing thread dump "Thread-2 ( ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@12aba8be)" Id=39 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSync hronizer$ConditionObject@308332ca at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@308332ca at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor$ThreadPoolQueue.poll(ActiveMQThreadPoolExecutor.java:112) at org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor$ThreadPoolQueue.poll(ActiveMQThreadPoolExecutor.java:45) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-4 (ActiveMQ-scheduled-threads)" Id=38 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Connector-Scheduler-4ce1d6d0-1" Id=37 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3311755f at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3311755f at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) "DestroyJavaVM" Id=36 RUNNABLE "activemq-discovery-group-thread-dg-group10 (DiscoveryGroup-1591484430)" Id=35 RUNNABLE (in native) at java.net.PlainDatagramSocketImpl.receive0(Native Method) - locked java.net.PlainDatagramSocketImpl@1df10aee at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:143) - locked java.net.PlainDatagramSocketImpl@1df10aee at java.net.DatagramSocket.receive(DatagramSocket.java:812) - locked java.net.DatagramPacket@34cd6ef8 - locked java.net.MulticastSocket@6e622381 at org.apache.activemq.artemis.api.core.UDPBroadcastEndpointFactory$UDPBroadcastEndpoint.receiveBroadcast(UDPBroadcastEndpointFactory.java:152) at org.apache.activemq.artemis.core.cluster.DiscoveryGroup$DiscoveryRunnable.run(DiscoveryGroup.java:274) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-1 ( ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@12aba8be)" Id=34 TIMED_WAITING on java.lang.Object@4352ced7 at java.lang.Object.wait(Native Method) - waiting on java.lang.Object@4352ced7 at org.apache.activemq.artemis.core.cluster.DiscoveryGroup.waitForBroadcast(DiscoveryGroup.java:225) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.checkOnDiscovery(ServerLocatorImpl.java:796) - locked java.lang.Object@56e1089f at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.executeDiscovery(ServerLocatorImpl.java:763) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:660) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:554) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:533) at org.apache.activemq.artemis.core.server.cluster.BackupManager$BackupConnector$1.run(BackupManager.java:270) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$62/404648734.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@689fdcd2 "Thread-3 (ActiveMQ-scheduled-threads)" Id=33 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-2 (ActiveMQ-scheduled-threads)" Id=32 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-0 ( ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$7@4659191b)" Id=31 TIMED_WAITING on java.util.concurrent.SynchronousQueue$Tran sferStack@1c37c2eb at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@1c37c2eb at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Session-HouseKeeper-48a12036-1" Id=30 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@63b694c7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@63b694c7 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) "qtp369347944-29" Id=29 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.lang.Thread.run(Thread.java:750) "qtp369347944-28" Id=28 TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@68bc88c6 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@68bc88c6 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:312) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:377) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750) "qtp369347944-27" Id=27 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.lang.Thread.run(Thread.java:750) "qtp369347944-26" Id=26 RUNNABLE (in native) at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked sun.nio.ch.Util$3@4a3e3fa0 - locked java.util.Collections$UnmodifiableSet@1d799a8c - locked sun.nio.ch.EPollSelectorImpl@4e0987d0 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750) "qtp369347944-25" Id=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.lang.Thread.run(Thread.java:750) "qtp369347944-24" Id=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.lang.Thread.run(Thread.java:750) qtp369347944-23-acceptor-0@5c3dde85-ServerConnector@4ce1d6d0{HTTP/1.1, (http/1.1)}{0.0.0.0:8161} Id=23 RUNNABLE (in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) - locked java.lang.Object@427fdc6f at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750) "qtp369347944-22" Id=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3dcdddde at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.lang.Thread.run(Thread.java:750) "activemq-discovery-group-thread-dg-group10 (DiscoveryGroup-500800186)" Id=21 RUNNABLE at java.net.PlainDatagramSocketImpl.receive0(Native Method) - locked java.net.PlainDatagramSocketImpl@798728bf at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:143) - locked java.net.PlainDatagramSocketImpl@798728bf at java.net.DatagramSocket.receive(DatagramSocket.java:812) - locked java.net.DatagramPacket@287b70f9 - locked java.net.MulticastSocket@2ace4521 at org.apache.activemq.artemis.api.core.UDPBroadcastEndpointFactory$UDPBroadcastEndpoint.receiveBroadcast(UDPBroadcastEndpointFactory.java:152) at org.apache.activemq.artemis.core.cluster.DiscoveryGroup$DiscoveryRunnable.run(DiscoveryGroup.java:274) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-1 (ActiveMQ-scheduled-threads)" Id=20 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "Thread-0 ( ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@12aba8be)" Id=19 RUNNABLE at sun.management.ThreadImpl.dumpThreads0(Native Method) at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:496) at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:484) at org.apache.activemq.artemis.utils.ThreadDumpUtil.threadDump(ThreadDumpUtil.java:47) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.checkOnDiscovery(ServerLocatorImpl.java:799) - locked java.lang.Object@16ee68b3 at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.executeDiscovery(ServerLocatorImpl.java:763) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:660) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:554) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:533) at org.apache.activemq.artemis.core.server.cluster.ClusterController$ConnectRunnable.run(ClusterController.java:497) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$62/404648734.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@1fd67f80 "Thread-0 (ActiveMQ-scheduled-threads)" Id=18 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6407de4e at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "PostgreSQL-JDBC-SharedTimer-1" Id=17 TIMED_WAITING on java.util.TaskQueue@566c7801 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@566c7801 at java.util.TimerThread.mainLoop(Timer.java:552) at java.util.TimerThread.run(Timer.java:505) "AMQ229000: Activation for server ActiveMQServerImpl::name=queue-b" Id=16 TIMED_WAITING at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at org.apache.activemq.artemis.core.server.impl.jdbc.LeaseLock$Pauser.lambda$sleep$0(LeaseLock.java:53) at org.apache.activemq.artemis.core.server.impl.jdbc.LeaseLock$Pauser$$Lambda$54/1090541608.idle(Unknown Source) at org.apache.activemq.artemis.core.server.impl.jdbc.LeaseLock.tryAcquire(LeaseLock.java:96) at org.apache.activemq.artemis.core.server.impl.jdbc.LeaseLock.tryAcquire(LeaseLock.java:111) at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.lock(JdbcNodeManager.java:284) at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.lockLiveAndCheckLiveState(JdbcNodeManager.java:354) at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:405) at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:92) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:4292) "PostgreSQL-JDBC-Cleaner" Id=15 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@2563368d at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@2563368d at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) at org.postgresql.util.LazyCleaner$1.run(LazyCleaner.java:128) at java.lang.Thread.run(Thread.java:750) "Thread-0 (-scheduled-threads)" Id=14 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d5c884d at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d5c884d at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) "ActiveMQ Artemis Server Shutdown Timer" Id=10 TIMED_WAITING on java.util.TaskQueue@1892dca5 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@1892dca5 at java.util.TimerThread.mainLoop(Timer.java:552) at java.util.TimerThread.run(Timer.java:505) "Signal Dispatcher" Id=5 RUNNABLE "Finalizer" Id=3 WAITING on java.lang.ref.ReferenceQueue$Lock@2d0b03c6 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@2d0b03c6 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:188) "Reference Handler" Id=2 WAITING on java.lang.ref.Reference$Lock@11608fb4 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.Reference$Lock@11608fb4 at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) =============================================================================== End Thread dump Discovery timeout, printing thread dump ******************************************************************************* Can anyone share any pointers to fix the same Thanks, Shivang. -- *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions maybe unlawful*. This e-mail may contain viruses. Provenir has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Provenir reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Provenir e-mail system.
