Jason, There should not be a host-specific message header that makes a persistent message belong to HOST1. In a shared-file-system master/slave setup, HOST2 takes the Store lock and recovers the same KahaDB. Once that recovery has completed successfully, the persistent messages should be both browseable and consumable on HOST2; no separate visibility setting should normally be required.
The important distinction is that an empty Web Console browse page does not by itself show that delivery has failed. I would test the paths separately: 1. Connect a small JMS QueueBrowser directly to HOST2, bypassing the Web Console. Also try one normal test consumer if that is safe for the queue. 2. Inspect the HOST2 queue MBean directly. Compare QueueSize, EnqueueCount, DequeueCount, DispatchCount, InFlightCount and ConsumerCount, and use the MBean browse operation if available. 3. Check the HOST2 startup log from the point where it acquires the KahaDB lock. Look for incomplete recovery, missing journal references, page/index warnings, checksum or corruption messages, and NFS I/O or lock errors. 4. Confirm that both brokers resolve the KahaDB directory to the same canonical NFS path, that only one broker has the lock at a time, and that HOST1 has fully released it before HOST2 starts recovery. ActiveMQ's shared-store documentation specifically depends on reliable exclusive file locking: https://activemq.apache.org/components/classic/documentation/shared-file-system-master-slave 5. Record one or two JMSMessageIDs before failover. After HOST2 is active, look for those exact IDs through the direct JMS/JMX browse path and, separately, verify whether a consumer can receive them. Those results divide the problem fairly cleanly: - A normal consumer receives the messages and JMX/JMS browse can see them, but the Web Console cannot: this is a Web Console presentation/browse path problem. - A consumer receives them but browse does not: delivery is working and the problem is specific to browsing. - QueueSize remains non-zero, direct browse cannot find the expected IDs, and consumers cannot receive them: investigate HOST2's KahaDB recovery/cursor state and the NFS lock/I/O transition. Since 6.2.1, later 6.2.x releases have included Web Console browse changes as well as other fixes. Reproducing on a current supported 6.2.x build is worthwhile, but the browse change I found was mainly about response content type and XML escaping, so it is not evidence by itself that upgrading fixes this particular symptom: https://github.com/apache/activemq/pull/1942 For Store-level comparison, take a consistent copy only while the broker is stopped, or use a storage snapshot with equivalent consistency. Do not point another broker or an offline tool at the live shared KahaDB. I work in ESB/message-integration support and built and maintain an open-source tool called MQ Watcher after needing a read-only way to inspect this kind of persisted evidence: https://github.com/kutaelee/mq-watcher It can trace supported JMSMessageID evidence to journal records, ACK/remove records, transactions and byte offsets. Its broker-generated fixtures currently cover ActiveMQ Classic 5.13.5, 5.15.16 and 5.18.7, not 6.2.1, so on this Store I would treat the result only as additional evidence, not as a compatibility guarantee or a diagnosis. The most useful next data would be one JMSMessageID checked through both the direct HOST2 browse/consume path and a consistent Store copy, together with the HOST2 recovery log around lock acquisition. Regards, Kutae Lee --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://activemq.apache.org/contact
