camel-jms uses spring jms, so there are many users with this combo also with IBM MQ and XA. Look at the many options spring jms has to tune its polling, and also cache levels you can tweak. There may be some more idle options you can set to make it "sleep" longer when there are no messages. But you can't avoid that spring jms is designed to keep polling for new messages also when the queue is empty.
On Tue, Sep 3, 2019 at 7:04 PM Benoit Fortin <benoit.for...@gmail.com> wrote: > > We have a java application doing JMS subscriptions that is using Camel as > its JMS provider. > > The application is subscribing to a topic using XA. It consumes 1 message > in the queue, and then closes the XA transaction (each message is part of > an XA transaction). Then the application re-attaches itself to the topic to > start over the same process for each message. > > When there are no messages to process, the client waits for 45 seconds > (which is the XA transaction timeout) before closing the request and the XA > transaction and starting a new iteration. > > I have analyzed how this process is actually being done using a tcpdump, > and here is what I found: > > —— > > MQ Client --[XA_START]--> MQ Server > > MQ Client <--[XA_START_REPLY]-- MQ Server > > > MQ Client --[SPI (SUBSCRIBE)]--> MQ Server > > MQ Client <--[SPI REPLY]-- MQ Server > > > MQ Client --[REQUEST_MSGS]--> MQ Server > > MQ Client --[REQUEST_MSGS]--> MQ Server > > MQ Client <--[NOTIFICATION]-- MQ Server > > > MQ Client --[MQCLOSE]--> MQ Server > > MQ Client <--[MQCLOSE_REPLY]-- MQ Server > > > MQ Client --[XA_END]--> MQ Server > > MQ Client <--[XA_END_REPLY]-- MQ Server > > > MQ Client --[XA_COMMIT]--> MQ Server > > MQ Client <--[XA_COMMIT_REPLY]-- MQ Server > > —— > > This is process is on a durable subscription, so each of the iterations > (even when there is no message to consume) ends up generating I/O on the MQ > server (I think, mostly on the SYSTEM.DURABLE.SUBSCRIBER.QUEUE queue, where > MQ keeps trace of its subscribers). > > Having a somewhat high number of subscribers doing this process, the MQ > server ends up I/O bound, with +4000 IOPS on the MQ logs disks (even > outside of business hours, when there are no messages to consume). This > process also consumes ~3 CPU outside of business hours. > > I am a bit puzzled that using XA on pub/sub scales so bad, and I am > wondering if there is any way to implement this solution without doing so > much subscribe operations. -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
