Hi, *Environment Summary* - ActiveMQ 5.11.0 (but issue also happened with 5.9.0) - Master-slave shared filesystem using KahaDB on top of GlusterFS - Multiple application servers producing and consuming messages via Camel as well as Spring DMLCs using local transactions - Connections are pooled and use failover transport - Each app server and broker are on their own Amazon EC2 instance
*Problem* When we failover to the slave broker, either by killing or performing a clean shutdown of the master broker, we more often that not get duplicates messages delivered and some pending messages in the queues which are not consumed. *Observations* - Transport failover appears to be working correctly although we see "Session is closed" errors when adding the session back to the pool and "Producer is closed" which causes a Camel route which is trying to send a message to be rolled back - Duplicates are suppressed by the consumer if it was the same consumer that handled the original - The pending messages remain even after a long period if the slave remains the master - The pending messages are dispatched if we switch back to the failed broker (which causes the described problem to happen again) *Desired Outcome* We need a highly available ActiveMQ configuration with guaranteed delivery and no duplicates as we are dealing with financial transactions. The setup we have seems quite simple and from all I've read doesn't seem unusual and I expect others have the same use case. I'm hoping it's just a configuration issue but I'm open to suggestions for alternative approaches. *Configuration* Broker configuration which is the same on both broker servers -- activemq.xml <http://activemq.2283324.n4.nabble.com/file/n4691679/activemq.xml> Application Spring configuration for ActiveMQ and Spring DMLCs -- jms-config.xml <http://activemq.2283324.n4.nabble.com/file/n4691679/jms-config.xml> Application Spring configuration for Camel, which references the above JMS config -- service-messaging-config.xml <http://activemq.2283324.n4.nabble.com/file/n4691679/service-messaging-config.xml> Kind Regards, Ben -- View this message in context: http://activemq.2283324.n4.nabble.com/Master-slave-shared-FS-failover-resulting-in-pending-msgs-that-are-not-consumed-until-failed-broker-n-tp4691679.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
