As far as I know ActiveMQ5 doesn't do failback on the master-slave journal... and it doesn't have any protocol to sync the data between master and slave.
There is a small regression on the failback that we are dealing now... if you set the master as a backup it would work fine... I think your testcase is a bit non orthodox... TBH production guys usually don't use failback.. they keep the backup until they can get to a quiet period and then do the failback (or restart the system) under low load. I also second Tim Bain on your choice for JDBC. I actually always say this.. if you can use JDBC as a storage for messaging.. don't use messaging at all.. just store and retrieve from the Database. There's a JIRA open for Artemis on JDBC.. but usually those things are written because users want, not need it. On Tue, Oct 20, 2015 at 3:12 AM, Mihkel Nõges <mihkel.no...@transferwise.com> wrote: > Yes I saw that issue too and set myself as watcher of this when it was > created. I did not think it could be exactly the same as it is described to > present itself only in narrow timing related conditions. My case seems to > be much more broad and basic. Seems like nobody actually tried to set this > up in realistic situation. > > Do you know of any existing production deployments of Artemis (or hornetq) > with failover? I thought Artemis as based on hornetq should have it's > features as stable as last hornetq version. We have already used embedded > hornetq for some time happily. I think it would make a lot of sense to > grade the Artemis features publicly as what is their maturity and usage > statistics of each feature if known, so it would be easier to compare the > brokers even among the 3 variants of ActiveMQ family. > > I think it's more safe for us to start building our first messaging > features on ActiveMQ 5.12.1 with JDBC backed Master-Slave instead of > Artemis and switch to Artemis once it has become more stable and also our > needs for scalability have grown to make it reasonable. Right now it seems > there are still too big blockers which may threaten the stability of our > system in Artemis. > > I did not mean this letter to be in no means negative. In the opposite I > really hope Artemis would do well as it comes with such a great technical > foundation and elegant ideas. I think the best for Artemis would be to find > users that can trust it's features and improve it as they grow. This means > the nucleus of Artemis must be really solid and stable. > > BR! > Mihkel Nõges > > > > On 19 October 2015 at 22:15, Clebert Suconic <clebert.suco...@gmail.com> > wrote: > >> Looks related to me: >> >> https://issues.apache.org/jira/browse/ARTEMIS-256 >> >> >> >> On Mon, Oct 19, 2015 at 4:04 AM, Mihkel Nõges >> <mihkel.no...@transferwise.com> wrote: >> > Basic flow of getting unresponsive failback cluster: >> > Have machine with Ubuntu 14.04.3 >> > >> > 1. Install libaio1, Java 1.8.0_60, maven 3.3.3, download and extract >> > apache-artemis-1.1.0-bin >> > < >> http://www.eu.apache.org/dist/activemq/activemq-artemis/1.1.0/apache-artemis-1.1.0-bin.tar.gz >> > >> > in >> > /opt >> > 2. run $ mvn -Prelease install and $ mnv verify in >> > /opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback >> > SUCCESS >> > 3. Clean data folders and starts both servers manually: >> > $ >> > cd >> /opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target >> > $ rm -R server0/data/ >> > $ rm -R server1/data/ >> > $ server0/bin/artemis-service start >> > Starting artemis-service >> > artemis-service is now running (7154) >> > $ server1/bin/artemis-service start >> > Starting artemis-service >> > artemis-service is now running (7180) >> > 4. Kill master server and wait for slave to take over >> > $ kill -9 7154 >> > >> > $ tail -f server1/log/artemis.log >> > 08:52:54,798 INFO [org.apache.activemq.artemis.core.server] >> AMQ221043: >> > Protocol module found: [artemis-stomp-protocol]. Adding protocol >> support >> > for: STOMP >> > 08:53:02,145 INFO [org.apache.activemq.artemis.core.server] >> AMQ221109: >> > Apache ActiveMQ Artemis Backup Server version 1.1.0 [null] started, >> waiting >> > live to fail before it gets active >> > 08:53:03,582 INFO [org.apache.activemq.artemis.core.server] >> AMQ221024: >> > Backup server >> > ActiveMQServerImpl::serverUUID=64ddff0f-7636-11e5-bfa8-f5004e6195f0 is >> > synchronized with live-server. >> > 08:53:03,777 INFO [org.apache.activemq.artemis.core.server] >> AMQ221031: >> > backup announced >> > 08:55:59,292 INFO [org.apache.activemq.artemis.core.server] >> AMQ221037: >> > ActiveMQServerImpl::serverUUID=64ddff0f-7636-11e5-bfa8-f5004e6195f0 to >> > become 'live' >> > 08:55:59,302 WARN [org.apache.activemq.artemis.core.client] >> AMQ212004: >> > Failed to connect to server. >> > 08:55:59,778 INFO [org.apache.activemq.artemis.core.server] >> AMQ221003: >> > trying to deploy queue jms.queue.exampleQueue >> > 08:55:59,829 WARN [org.apache.activemq.artemis.core.client] >> AMQ212034: >> > There are more than one servers on the network broadcasting the same >> node >> > id. You will see this message exactly once (per node) if a node is >> > restarted, in which case it can be safely ignored. But if it is logged >> > continuously it means you really do have more than one node on the >> same >> > network active concurrently with the same node id. This could occur >> if you >> > have a backup node active at the same time as its live node. >> > nodeID=64ddff0f-7636-11e5-bfa8-f5004e6195f0 >> > 08:55:59,836 INFO [org.apache.activemq.artemis.core.server] >> AMQ221007: >> > Server is now live >> > 08:55:59,869 INFO [org.apache.activemq.artemis.core.server] >> AMQ221020: >> > Started Acceptor at broker3:61617 for protocols >> > [CORE,MQTT,AMQP,HORNETQ,STOMP,OPENWIRE] >> > 5. >> > >> > Start master again and observer the logs: >> > $ server0/bin/artemis-service start >> > Starting artemis-service >> > artemis-service is now running (7388) >> > >> > $ tail -f server0/log/artemis.log >> > 08:57:24,625 INFO [org.apache.activemq.artemis.core.server] AMQ221012: >> > Using AIO Journal >> > 08:57:24,694 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-server]. Adding protocol support for: >> CORE >> > 08:57:24,702 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-amqp-protocol]. Adding protocol support >> > for: AMQP >> > 08:57:24,731 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-hornetq-protocol]. Adding protocol >> support >> > for: HORNETQ >> > 08:57:24,733 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-mqtt-protocol]. Adding protocol support >> > for: MQTT >> > 08:57:24,743 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-openwire-protocol]. Adding protocol >> support >> > for: OPENWIRE >> > 08:57:24,878 INFO [org.apache.activemq.artemis.core.server] AMQ221043: >> > Protocol module found: [artemis-stomp-protocol]. Adding protocol support >> > for: STOMP >> > 08:57:25,082 INFO [org.apache.activemq.artemis.core.server] AMQ221109: >> > Apache ActiveMQ Artemis Backup Server version 1.1.0 [null] started, >> waiting >> > live to fail before it gets active >> > 08:57:27,043 INFO [org.apache.activemq.artemis.core.server] AMQ221024: >> > Backup server >> > ActiveMQServerImpl::serverUUID=64ddff0f-7636-11e5-bfa8-f5004e6195f0 is >> > synchronized with live-server. >> > 08:57:27,948 INFO [org.apache.activemq.artemis.core.server] AMQ221031: >> > backup announced >> > 08:57:31,227 WARN [org.apache.activemq.artemis.core.client] AMQ212037: >> > Connection failure has been detected: AMQ119015: The connection was >> > disconnected because of server shutdown [code=DISCONNECTED] >> > 08:57:31,252 WARN [org.apache.activemq.artemis.core.client] AMQ212037: >> > Connection failure has been detected: AMQ119015: The connection was >> > disconnected because of server shutdown [code=DISCONNECTED] >> > 08:57:31,307 WARN [org.apache.activemq.artemis.core.client] AMQ212037: >> > Connection failure has been detected: AMQ119015: The connection was >> > disconnected because of server shutdown [code=DISCONNECTED] >> > 08:57:31,339 INFO [org.apache.activemq.artemis.core.server] AMQ221037: >> > ActiveMQServerImpl::serverUUID=64ddff0f-7636-11e5-bfa8-f5004e6195f0 to >> > become 'live' >> > 08:57:31,360 WARN [org.apache.activemq.artemis.core.client] AMQ212004: >> > Failed to connect to server. >> > 08:57:31,413 ERROR [org.apache.activemq.artemis.core.server] AMQ224008: >> > Failed to store id: java.lang.IllegalStateException: Cannot find add >> info 1 >> > at >> > >> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendDeleteRecord(JournalImpl.java:799) >> > [artemis-journal-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.journal.impl.JournalBase.appendDeleteRecord(JournalBase.java:183) >> > [artemis-journal-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendDeleteRecord(JournalImpl.java:79) >> > [artemis-journal-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.deleteID(JournalStorageManager.java:1194) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.persistence.impl.journal.BatchingIDGenerator.deleteID(BatchingIDGenerator.java:152) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.persistence.impl.journal.BatchingIDGenerator.cleanup(BatchingIDGenerator.java:75) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadBindingJournal(JournalStorageManager.java:1784) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1625) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1535) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run(SharedNothingBackupActivation.java:249) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_60] >> > 08:57:31,540 WARN [org.apache.activemq.artemis.core.server] AMQ222173: >> > Queue jms.queue.exampleQueue is duplicated during reload. This queue will >> > be renamed as jms.queue.exampleQueue-0 >> > 08:57:31,550 ERROR [org.apache.activemq.artemis.core.server] AMQ224000: >> > Failure in initialisation: java.lang.IllegalStateException: Cursor 2 had >> > already been created >> > at >> > >> org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.createSubscription(PageCursorProviderImpl.java:97) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.PostOfficeJournalLoader.initQueues(PostOfficeJournalLoader.java:140) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1631) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1535) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at >> > >> org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run(SharedNothingBackupActivation.java:249) >> > [artemis-server-1.1.0.jar:1.1.0] >> > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_60] >> > >> > >> > On 19 October 2015 at 10:31, Mihkel Nõges <mihkel.no...@transferwise.com >> > >> > wrote: >> > >> >> Hi Clebert, >> >> >> >> I do not have other code to share with you but the example code in >> Artemis >> >> 1.1.0 binary deployment package. I'm running >> >> org.apache.activemq.artemis.jms.example.ReplicatedFailbackExample >> >> >> >> And only commented out the serverStart and killServer calls which I am >> >> doing manually. >> >> >> >> I do not think I do any of the steps too fast as I tail the server log >> >> files in parallel and see everything is finished when I start the fail >> >> back. I have waited many minutes in between. >> >> >> >> Only changes in configuration to the test is changing localhost >> addresses >> >> with broker3 to make the cluster accessible remotely. >> >> >> >> BR! >> >> MIhkel >> >> >> >> On 18 October 2015 at 17:49, Clebert <clebert.suco...@gmail.com> wrote: >> >> >> >>> Im not on my computer now but it sounds like you are doing a fail back >> >>> immediately after failed over. It takes some time (seconds) to the >> server >> >>> to activate on the backup. >> >>> >> >>> Later the server will need to copy the data back before it can be >> >>> activated in fail back mode. >> >>> >> >>> It sounds the live is not reaching backup for fail back. >> >>> >> >>> I will try looking it at it on Monday. Maybe you could post your >> example >> >>> at your GitHub fork. >> >>> >> >>> -- Clebert Suconic typing on the iPhone. >> >>> >> >>> > On Oct 18, 2015, at 08:15, Mihkel Nõges < >> mihkel.no...@transferwise.com> >> >>> wrote: >> >>> > >> >>> > Hello again! >> >>> > >> >>> > I would be very grateful If someone could answer my questions. We >> need >> >>> the high availability to work to use the broker in production. >> >>> > >> >>> > When I run the replicated-failback example in one machine (broker3) >> it >> >>> succeeds. >> >>> > >> >>> > It fails when I run the same test - exactly the same servers with >> >>> slightly modified client remotely. >> >>> > >> >>> > I run client in debug mode from my IDE with commented out serverStart >> >>> and killServer calls. >> >>> > Deleted data folders and started the servers: >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> rm -R server0/data/ >> >>> > >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> rm -R server1/data/ >> >>> > >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> server0/bin/artemis-service start >> >>> > >> >>> > Starting artemis-service >> >>> > >> >>> > artemis-service is now running (23357) >> >>> > >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> server1/bin/artemis-service start >> >>> > >> >>> > Starting artemis-service >> >>> > >> >>> > artemis-service is now running (23383) >> >>> > >> >>> > Starting client and stopping on breakpoint at line 103: >> >>> > //ServerUtil.killServer(server0); >> >>> > // Step 11. Acknowledging the 2nd half of the sent messages will fail >> >>> as failover to the >> >>> > // backup server has occurred >> >>> > try { >> >>> > message0.acknowledge(); //line 103 >> >>> > killing server0 >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> kill -9 23357 >> >>> > >> >>> > Proceeding to breakpoint at line 121: >> >>> > //server0 = ServerUtil.startServer(args[0], >> >>> ReplicatedFailbackExample.class.getSimpleName() + "0", 0, 10000); >> >>> > >> >>> > // Step 11. Acknowledging the 2nd half of the sent messages will fail >> >>> as failover to the >> >>> > // backup server has occurred >> >>> > try { >> >>> > message0.acknowledge(); // line 121 >> >>> > Starting server0: >> >>> > artemis@broker3 >> :/opt/apache-artemis-1.1.0/examples/features/ha/replicated-failback/target$ >> >>> server0/bin/artemis-service start >> >>> > >> >>> > Starting artemis-service >> >>> > >> >>> > artemis-service is now running (24240) >> >>> > >> >>> > Server0 writes ERROR to it's log (see attached server0_artemis.log). >> >>> > Now when trying to proceed with the client it writes the following in >> >>> the log and does not exit, but remains hanging forever: >> >>> > >> >>> > Oct 18, 2015 2:55:34 PM >> >>> >> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl >> >>> fail >> >>> > >> >>> > WARN: AMQ212037: Connection failure has been detected: AMQ119015: The >> >>> connection was disconnected because of server shutdown >> [code=DISCONNECTED] >> >>> > >> >>> > Got message: This is text message 20 (redelivered?: false) >> >>> > >> >>> > Got exception while acknowledging message: AMQ119014: Timed out after >> >>> waiting 30,000 ms for response when sending packet 43 >> >>> > >> >>> > Got message: This is text message 21 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 22 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 23 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 24 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 25 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 26 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 27 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 28 (redelivered?: false) >> >>> > >> >>> > Got message: This is text message 29 (redelivered?: false) >> >>> > >> >>> > As a result the slave (server1) remains stopped, not restarted as >> >>> expected and the master (server0) process appears to be running but >> does >> >>> not accept any connections. >> >>> > >> >>> > Exactly the same behavior is observable every time I try this. >> >>> > >> >>> > BR! >> >>> > Mihkel >> >>> > >> >>> >> On 13 October 2015 at 20:17, Mihkel Nõges < >> >>> mihkel.no...@transferwise.com> wrote: >> >>> >> Hi Clebert, >> >>> >> >> >>> >> No test, just doing it on command line with standalone servers. I'm >> >>> using 1.1.0 installed with wget, not the snapshot. >> >>> >> >> >>> >> I'm wondering what should be the suggested procedure for admins to >> do >> >>> changes to HA cluster of 2 or 3 nodes of Artemis. If one of the nodes >> is >> >>> master by configuration, do they need to change it's config before >> >>> restarting it to become slave to have seamless change process and make >> some >> >>> instance master by configuration only if all the instances need to be >> >>> restarted? >> >>> >> >> >>> >> I tried also a cluster with 2 masters and 2 slaves with 2 separate >> >>> group-name values, but for some reason the second master I started >> became >> >>> slave for the first immediately. I expected it to become a clustered >> >>> load-balancing parallel master. Our loads are not yet that high to >> require >> >>> more than one master, so it's just a theoretical question. >> >>> >> >> >>> >> BR! >> >>> >> Mihkel >> >>> >> >> >>> >>> On 13 October 2015 at 20:03, Clebert Suconic < >> >>> clebert.suco...@gmail.com> wrote: >> >>> >>> The master needs to copy its data from the backup back to live >> before >> >>> >>> it's activated. >> >>> >>> >> >>> >>> Do you have a test replicating this? >> >>> >>> >> >>> >>> Did you try the snapshot build? >> >>> >>> >> >>> >>> On Tue, Oct 13, 2015 at 11:58 AM, Mihkel Nõges >> >>> >>> <mihkel.no...@transferwise.com> wrote: >> >>> >>> > Hi, >> >>> >>> > >> >>> >>> > I configured replicating HA master-slave of Artemis 1.1.0 >> instances >> >>> on >> >>> >>> > Ubuntu 14.04.3. >> >>> >>> > >> >>> >>> > When I kill master the slave takes over as expected and starts >> >>> serving as >> >>> >>> > new master. When I then start the old master, it fails with the >> >>> following >> >>> >>> > errors in the log: >> >>> >>> > >> >>> >>> > 16:35:46,476 ERROR [org.apache.activemq.artemis.core.server] >> >>> AMQ224008: >> >>> >>> > Failed to store id: java.lang.IllegalStateException: Cannot find >> >>> add info 1 >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendDeleteRecord(JournalImpl.java:799) >> >>> >>> > [artemis-journal-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.journal.impl.JournalBase.appendDeleteRecord(JournalBase.java:183) >> >>> >>> > [artemis-journal-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendDeleteRecord(JournalImpl.java:79) >> >>> >>> > [artemis-journal-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.deleteID(JournalStorageManager.java:1194) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.persistence.impl.journal.BatchingIDGenerator.deleteID(BatchingIDGenerator.java:152) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.persistence.impl.journal.BatchingIDGenerator.cleanup(BatchingIDGenerator.java:75) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadBindingJournal(JournalStorageManager.java: >> >>> 1784) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java: >> >>> 1625) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java: >> >>> 1535) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run(SharedNothingBackupActivation.java:249) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_60] >> >>> >>> > >> >>> >>> > 16:35:46,572 WARN [org.apache.activemq.artemis.core.server] >> >>> AMQ222173: >> >>> >>> > Queue jms.queue.DLQ is duplicated during reload. This queue will >> be >> >>> renamed >> >>> >>> > as jms.queue.DLQ-0 >> >>> >>> > 16:35:46,572 ERROR [org.apache.activemq.artemis.core.server] >> >>> AMQ224000: >> >>> >>> > Failure in initialisation: java.lang.IllegalStateException: >> Cursor >> >>> 2 had >> >>> >>> > already been created >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.createSubscription(PageCursorProviderImpl.java:97) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.PostOfficeJournalLoader.initQueues(PostOfficeJournalLoader.java:140) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java: >> >>> 1631) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java: >> >>> 1535) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at >> >>> >>> > >> >>> >> org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run(SharedNothingBackupActivation.java:249) >> >>> >>> > [artemis-server-1.1.0.jar:1.1.0] >> >>> >>> > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_60] >> >>> >>> > >> >>> >>> > As a result both master and the slave remain unaccessible and no >> >>> further >> >>> >>> > restarts solve the situation. >> >>> >>> > >> >>> >>> > Attached also master and slave broker.xml files. >> >>> >>> > >> >>> >>> > BR! >> >>> >>> > >> >>> >>> > Mihkel Nõges >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> -- >> >>> >>> Clebert Suconic >> >>> > >> >>> >> >> >> >> >> >> >> >> -- >> Clebert Suconic >> -- Clebert Suconic