Are both the source and destination queues hosted by the same ActiveMQ broker?
> On Feb 3, 2016, at 8:21 AM, Stephan Burkard <sburk...@gmail.com> wrote: > > Hi > > I have built a small Maven project (attached) to demonstrate a JMS > transaction problem in Camel routes under certain load conditions. In fact I > am losing messages between two queues. > > The project contains two different flavours of the same test. One of them > suffers from the problem, the other (due to my tests) not. > > > *** What does the testcase? > 1. Produces 1000 messages (100/s) and sends them to an "input" queue. > 2. Sends the messages from the "input" queue to an "output" queue. > 3. Finally consumes the messages from the "output" queue to count them. > > > *** What is the difference between the two test flavours? > - There is a "standard" flavour that suffers from the problem > - And there is a "noTxManager" flavour that seems to not have the problem > - The "standard" flavour is kind of a well known Camel/ActiveMQ configuration > - with a Spring transaction manager > - with a Spring transaction policy > - With a "transacted" flag in Camel routes > - The "noTxManager" flavour is a "simple" configuration > - no Spring transaction manager > - no Spring transaction policy > - no "transacted" flag in Camel routes > - BUT: "lazyCreateTransactionManager" = false (so routes are transacted too) > > > *** How to run the testcases? > 1. Replace "[yourBrokerHost]" with the hostname of your ActiveMQ broker > 2. Run the testcase as JUnit test > 3. When you see lots of console messages that messages are sent, stop your > ActiveMQ broker (do not kill-9 it, just shut it down normally) > 4. Exceptions are thrown on the console output > 5. After some seconds start your broker again > 6. The test finish normally and after some seconds dumps a book keeping on > the console > > > *** How to interpret the results? > - When the test is successful, no message is lost. You can run the test > without broker shutdown/startup and it will obviously always be successful. > - When the test fails, one or more messages are lost between queue "input" > and "output". In my tests I was not able to run the "standard" flavour three > times in a row successfully. About every second run failed. In contrast, the > "noTxManager" flavour never failed in my tests. > > The book keeping for a failed test looks like the following. In this example > Message number 281 is arrived at the input queue but not at the output queue. > So it is lost. > > Messages created by Client: 1000 > Client Exceptions during send: 0 [] > > Messages received at input queue: 993 > Missing Messages at input queue: 7 [282,283,284,285,286,287,288] > Duplicate Messages at input queue: 0 [] > > Messages received at output queue: 992 > Missing Messages at output queue: 8 [281,282,283,284,285,286,287,288] > Duplicate Messages at output queue: 0 [] > > Lost Messages between Queues: 1 [281] > > > *** What is the problem? > A Redhat engineer tracked the problem down to a Spring JMS template behaviour > that is kind of strange. If a Spring transaction manager is defined in the > config, it will end up with two of them. Therefore the small time range where > messages can get lost that arises only when you have a certain load. > > > *** So, what is my question? > - Does this really mean that it is unsafe to use the "standard" flavour of > configuration? > - Is there another config with TxManager etc that works correctly? > - What are limits of the "noTxManager" config? When is it not sufficent? > > Regards > Stephan > > > > > <CamelAmqTxTest.zip>
