I tested this with a 5.9.0 broker and I am seeing messages dropped with the TxText, but I still have to use the failover URL or the test just stops after the broker is restarted.
I don’t have a 5.9.1 broker to test with, so I don’t know if that would help, but the next oldest broker I have is 5.10.1, and it seems to be working with that broker. NOTE: I’m not changing the activemq-version in the POM when I change the broker version - I’m just starting a different broker (locally) on the same port. > On Feb 4, 2016, at 10:41 AM, Quinn Stevenson <qu...@pronoia-solutions.com> > wrote: > > I still can’t make either test drop messages between the input and the output > queue with the POM changes I sent, but I did find one difference between what > you’ve done and what I normally do that changes the output I’m seeing - I > always use a failover URL > > <property name="brokerURL" > value="failover:(tcp://localhost:61616?wireFormat.tightEncodingEnabled=false > <tcp://localhost:61616?wireFormat.tightEncodingEnabled=false>)"/> > > My test broker is v 5.10.1 as well - I’ll see if it makes any difference with > 5.9.0 > > > >> On Feb 4, 2016, at 9:52 AM, Quinn Stevenson <qu...@pronoia-solutions.com >> <mailto:qu...@pronoia-solutions.com>> wrote: >> >> It is strange - I’m trying to compare what you have in the “standard” >> version to what I did before. We tested our configs pretty heavily under >> all sorts of strange conditions to verify we weren’t looking messages, but >> we were using newer versions of Camel and ActiveMQ. >> >> So we’re on the same page - can you try your tests again with POM >> dependencies that look something like this? >> >> <properties> >> <camel-version>2.12.5</camel-version> >> <activemq-version>5.9.0</activemq-version> >> </properties> >> >> <dependencies> >> <dependency> >> <groupId>org.apache.activemq</groupId> >> <artifactId>activemq-all</artifactId> >> <version>${activemq-version}</version> >> </dependency> >> <dependency> >> <groupId>org.apache.activemq</groupId> >> <artifactId>activemq-pool</artifactId> >> <version>${activemq-version}</version> >> </dependency> >> >> <dependency> >> <groupId>org.apache.camel</groupId> >> <artifactId>camel-spring</artifactId> >> <version>${camel-version}</version> >> </dependency> >> <dependency> >> <groupId>org.apache.camel</groupId> >> <artifactId>camel-jms</artifactId> >> <version>${camel-version}</version> >> </dependency> >> >> <dependency> >> <groupId>org.apache.camel</groupId> >> <artifactId>camel-test-spring</artifactId> >> <version>${camel-version}</version> >> <scope>test</scope> >> </dependency> >> >> <dependency> >> <groupId>commons-collections</groupId> >> <artifactId>commons-collections</artifactId> >> <version>3.2.1</version> >> <scope>test</scope> >> </dependency> >> <dependency> >> <groupId>org.hamcrest</groupId> >> <artifactId>hamcrest-integration</artifactId> >> <version>1.3</version> >> <scope>test</scope> >> </dependency> >> >> </dependencies> >> >> >> >>> On Feb 4, 2016, at 9:49 AM, Stephan Burkard <sburk...@gmail.com >>> <mailto:sburk...@gmail.com>> wrote: >>> >>> Hi Quinn >>> >>> The "standard" version is the big mystery. As I stated in my first post, a >>> Redhat engineer analysed a similar project (with less book-keeping and >>> logging stuff) and his conclusion was that as soon as a transaction manager >>> is explicitly defined, Spring JMS Template (that is used by Camel under the >>> hood) creates two of them by bug, by accident or just by strange behaviour. >>> >>> This conclusion was quite suprising since that meant that all our Camel-JMS >>> project are theoretically suffering from message loss. >>> >>> The "no-tx" version should definitely be OK, see also CAMEL-5055 for the " >>> lazyCreateTransactionManager" flag. The JMS transaction manager may not be >>> defined but it creates one implicitly because of "transacted = true". >>> >>> The two "flaws" you mentioned are perhaps an issue. It would be somehow >>> calming if it is my project who has a flaw. >>> >>> Regards >>> Stephan >>> >>> >>> >>> On Thu, Feb 4, 2016 at 4:44 PM, Quinn Stevenson >>> <qu...@pronoia-solutions.com <mailto:qu...@pronoia-solutions.com> >>>> wrote: >>> >>>> I’m still going through the project, but the first couple of things that >>>> jump out at me are you have two Spring versions - the one you explicitly >>>> put in your POM (3.2.8.RELEASE) and the one pulled in by camel-spring >>>> (3.2.11.RELEASE). Also, camel-spring should be included in the POM since >>>> you’re using Spring routes. I’m not sure if that’s enough to cause issues >>>> or not. >>>> >>>> I believe what’s going on with the “no-tx” version is you’re actually >>>> using JMS transactions since you still have transacted set to true in the >>>> JmsConfiguration. >>>> >>>> I’m not sure what’s going in with the “standard” version - it looks >>>> similar to some XA stuff I’ve setup before (because I had multiple brokers >>>> involved) except I had to use XA Connection Factories. >>>> >>>> >>>> >>>>> On Feb 3, 2016, at 3:12 PM, Stephan Burkard <sburk...@gmail.com >>>>> <mailto:sburk...@gmail.com>> wrote: >>>>> >>>>> Yes, same broker. There is only one ActiveMQ connection config in the >>>>> project. >>>>> >>>>> >>>>> >>>>> On Wed, Feb 3, 2016 at 8:00 PM, Quinn Stevenson < >>>> qu...@pronoia-solutions.com <mailto:qu...@pronoia-solutions.com> >>>>>> wrote: >>>>> >>>>>> Are both the source and destination queues hosted by the same ActiveMQ >>>>>> broker? >>>>>> >>>>>> >>>>>> >>>>>>> On Feb 3, 2016, at 8:21 AM, Stephan Burkard <sburk...@gmail.com >>>>>>> <mailto:sburk...@gmail.com>> >>>> wrote: >>>>>>> >>>>>>> Hi >>>>>>> >>>>>>> I have built a small Maven project (attached) to demonstrate a JMS >>>>>> transaction problem in Camel routes under certain load conditions. In >>>> fact >>>>>> I am losing messages between two queues. >>>>>>> >>>>>>> The project contains two different flavours of the same test. One of >>>>>> them suffers from the problem, the other (due to my tests) not. >>>>>>> >>>>>>> >>>>>>> *** What does the testcase? >>>>>>> 1. Produces 1000 messages (100/s) and sends them to an "input" queue. >>>>>>> 2. Sends the messages from the "input" queue to an "output" queue. >>>>>>> 3. Finally consumes the messages from the "output" queue to count them. >>>>>>> >>>>>>> >>>>>>> *** What is the difference between the two test flavours? >>>>>>> - There is a "standard" flavour that suffers from the problem >>>>>>> - And there is a "noTxManager" flavour that seems to not have the >>>> problem >>>>>>> - The "standard" flavour is kind of a well known Camel/ActiveMQ >>>>>> configuration >>>>>>> - with a Spring transaction manager >>>>>>> - with a Spring transaction policy >>>>>>> - With a "transacted" flag in Camel routes >>>>>>> - The "noTxManager" flavour is a "simple" configuration >>>>>>> - no Spring transaction manager >>>>>>> - no Spring transaction policy >>>>>>> - no "transacted" flag in Camel routes >>>>>>> - BUT: "lazyCreateTransactionManager" = false (so routes are >>>>>> transacted too) >>>>>>> >>>>>>> >>>>>>> *** How to run the testcases? >>>>>>> 1. Replace "[yourBrokerHost]" with the hostname of your ActiveMQ broker >>>>>>> 2. Run the testcase as JUnit test >>>>>>> 3. When you see lots of console messages that messages are sent, stop >>>>>> your ActiveMQ broker (do not kill-9 it, just shut it down normally) >>>>>>> 4. Exceptions are thrown on the console output >>>>>>> 5. After some seconds start your broker again >>>>>>> 6. The test finish normally and after some seconds dumps a book keeping >>>>>> on the console >>>>>>> >>>>>>> >>>>>>> *** How to interpret the results? >>>>>>> - When the test is successful, no message is lost. You can run the test >>>>>> without broker shutdown/startup and it will obviously always be >>>> successful. >>>>>>> - When the test fails, one or more messages are lost between queue >>>>>> "input" and "output". In my tests I was not able to run the "standard" >>>>>> flavour three times in a row successfully. About every second run >>>> failed. >>>>>> In contrast, the "noTxManager" flavour never failed in my tests. >>>>>>> >>>>>>> The book keeping for a failed test looks like the following. In this >>>>>> example Message number 281 is arrived at the input queue but not at the >>>>>> output queue. So it is lost. >>>>>>> >>>>>>> Messages created by Client: 1000 >>>>>>> Client Exceptions during send: 0 [] >>>>>>> >>>>>>> Messages received at input queue: 993 >>>>>>> Missing Messages at input queue: 7 [282,283,284,285,286,287,288] >>>>>>> Duplicate Messages at input queue: 0 [] >>>>>>> >>>>>>> Messages received at output queue: 992 >>>>>>> Missing Messages at output queue: 8 >>>> [281,282,283,284,285,286,287,288] >>>>>>> Duplicate Messages at output queue: 0 [] >>>>>>> >>>>>>> Lost Messages between Queues: 1 [281] >>>>>>> >>>>>>> >>>>>>> *** What is the problem? >>>>>>> A Redhat engineer tracked the problem down to a Spring JMS template >>>>>> behaviour that is kind of strange. If a Spring transaction manager is >>>>>> defined in the config, it will end up with two of them. Therefore the >>>> small >>>>>> time range where messages can get lost that arises only when you have a >>>>>> certain load. >>>>>>> >>>>>>> >>>>>>> *** So, what is my question? >>>>>>> - Does this really mean that it is unsafe to use the "standard" flavour >>>>>> of configuration? >>>>>>> - Is there another config with TxManager etc that works correctly? >>>>>>> - What are limits of the "noTxManager" config? When is it not >>>> sufficent? >>>>>>> >>>>>>> Regards >>>>>>> Stephan >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> <CamelAmqTxTest.zip> >>>>>> >>>>>> >>>> >>>> >> >
