Hi all, We've been investigating split-brain behavior with ActiveMQ Classic 6.2.7 using KahaDB's shared-file-lock mechanism over a hard-mounted NFSv4 share (AWS FSx), and found what looks like a separate issue from AMQ-5568.
Summary: When the NFS server becomes unreachable while hard-mounted, LockFile.hasBeenModified() (called from SharedFileLocker.keepAlive()) can block indefinitely in the underlying native stat() syscall, since a hard NFS mount deliberately never returns an error — it waits for the server to come back. Because keepAlive() itself never returns, the broker never reaches the stopOnError/stopBroker() path that's supposed to detect lock loss. The broker keeps running as a "zombie master": its transport connectors stay open and keep accepting connections, while the standby broker (once the NFSv4 client lease/grace period elapses) can acquire the lock and become a second, independent master. How this differs from AMQ-5568: That issue was about deleting the lock file during shutdown potentially bringing down the new master too, and was fixed by making deleteOnUnlock always false. That fix doesn't touch this issue, since in our case keepAlive() hangs before any shutdown/delete path is ever reached. Reproduction (locally, Docker Compose, two brokers sharing one NFSv4-mounted KahaDB store): * Simulate NFS server unreachability for the current master only (iptables DROP on ports 2049/111 from that container's network namespace, not the NFS server itself) * Thread dump (SIGQUIT, since jstack isn't in the JRE-only image) shows multiple "ActiveMQ Lock KeepAlive Timer" threads stuck in: at java.io.UnixFileSystem.getBooleanAttributes0(Native Method) at java.io.UnixFileSystem.hasBooleanAttributes at java.io.File.exists at org.apache.activemq.util.LockFile.hasBeenModified(LockFile.java:167) at org.apache.activemq.util.LockFile.keepAlive(LockFile.java:183) at org.apache.activemq.store.SharedFileLocker.keepAlive(SharedFileLocker.java:97) at org.apache.activemq.broker.LockableServiceSupport.keepLockAlive(LockableServiceSupport.java:137) * Independently confirmed: timeout 5 stat <kahadb-dir> from the host returns exit code 124 (timeout), not an error, for the whole duration of the outage. We don't currently have a JIRA account - wanted to check here first whether this is a known/expected limitation, whether it's worth a new ticket, or whether we're missing an existing configuration option that addresses it, before filing anything. Thanks, Axel
