I still can’t make either test drop messages between the input and the output
queue with the POM changes I sent, but I did find one difference between what
you’ve done and what I normally do that changes the output I’m seeing - I
always use a failover URL
<property name="brokerURL"
value="failover:(tcp://localhost:61616?wireFormat.tightEncodingEnabled=false)"/>
My test broker is v 5.10.1 as well - I’ll see if it makes any difference with
5.9.0
> On Feb 4, 2016, at 9:52 AM, Quinn Stevenson <qu...@pronoia-solutions.com>
> wrote:
>
> It is strange - I’m trying to compare what you have in the “standard” version
> to what I did before. We tested our configs pretty heavily under all sorts
> of strange conditions to verify we weren’t looking messages, but we were
> using newer versions of Camel and ActiveMQ.
>
> So we’re on the same page - can you try your tests again with POM
> dependencies that look something like this?
>
> <properties>
> <camel-version>2.12.5</camel-version>
> <activemq-version>5.9.0</activemq-version>
> </properties>
>
> <dependencies>
> <dependency>
> <groupId>org.apache.activemq</groupId>
> <artifactId>activemq-all</artifactId>
> <version>${activemq-version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.activemq</groupId>
> <artifactId>activemq-pool</artifactId>
> <version>${activemq-version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.camel</groupId>
> <artifactId>camel-spring</artifactId>
> <version>${camel-version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.camel</groupId>
> <artifactId>camel-jms</artifactId>
> <version>${camel-version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.camel</groupId>
> <artifactId>camel-test-spring</artifactId>
> <version>${camel-version}</version>
> <scope>test</scope>
> </dependency>
>
> <dependency>
> <groupId>commons-collections</groupId>
> <artifactId>commons-collections</artifactId>
> <version>3.2.1</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.hamcrest</groupId>
> <artifactId>hamcrest-integration</artifactId>
> <version>1.3</version>
> <scope>test</scope>
> </dependency>
>
> </dependencies>
>
>
>
>> On Feb 4, 2016, at 9:49 AM, Stephan Burkard <sburk...@gmail.com
>> <mailto:sburk...@gmail.com>> wrote:
>>
>> Hi Quinn
>>
>> The "standard" version is the big mystery. As I stated in my first post, a
>> Redhat engineer analysed a similar project (with less book-keeping and
>> logging stuff) and his conclusion was that as soon as a transaction manager
>> is explicitly defined, Spring JMS Template (that is used by Camel under the
>> hood) creates two of them by bug, by accident or just by strange behaviour.
>>
>> This conclusion was quite suprising since that meant that all our Camel-JMS
>> project are theoretically suffering from message loss.
>>
>> The "no-tx" version should definitely be OK, see also CAMEL-5055 for the "
>> lazyCreateTransactionManager" flag. The JMS transaction manager may not be
>> defined but it creates one implicitly because of "transacted = true".
>>
>> The two "flaws" you mentioned are perhaps an issue. It would be somehow
>> calming if it is my project who has a flaw.
>>
>> Regards
>> Stephan
>>
>>
>>
>> On Thu, Feb 4, 2016 at 4:44 PM, Quinn Stevenson <qu...@pronoia-solutions.com
>> <mailto:qu...@pronoia-solutions.com>
>>> wrote:
>>
>>> I’m still going through the project, but the first couple of things that
>>> jump out at me are you have two Spring versions - the one you explicitly
>>> put in your POM (3.2.8.RELEASE) and the one pulled in by camel-spring
>>> (3.2.11.RELEASE). Also, camel-spring should be included in the POM since
>>> you’re using Spring routes. I’m not sure if that’s enough to cause issues
>>> or not.
>>>
>>> I believe what’s going on with the “no-tx” version is you’re actually
>>> using JMS transactions since you still have transacted set to true in the
>>> JmsConfiguration.
>>>
>>> I’m not sure what’s going in with the “standard” version - it looks
>>> similar to some XA stuff I’ve setup before (because I had multiple brokers
>>> involved) except I had to use XA Connection Factories.
>>>
>>>
>>>
>>>> On Feb 3, 2016, at 3:12 PM, Stephan Burkard <sburk...@gmail.com
>>>> <mailto:sburk...@gmail.com>> wrote:
>>>>
>>>> Yes, same broker. There is only one ActiveMQ connection config in the
>>>> project.
>>>>
>>>>
>>>>
>>>> On Wed, Feb 3, 2016 at 8:00 PM, Quinn Stevenson <
>>> qu...@pronoia-solutions.com <mailto:qu...@pronoia-solutions.com>
>>>>> wrote:
>>>>
>>>>> Are both the source and destination queues hosted by the same ActiveMQ
>>>>> broker?
>>>>>
>>>>>
>>>>>
>>>>>> On Feb 3, 2016, at 8:21 AM, Stephan Burkard <sburk...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I have built a small Maven project (attached) to demonstrate a JMS
>>>>> transaction problem in Camel routes under certain load conditions. In
>>> fact
>>>>> I am losing messages between two queues.
>>>>>>
>>>>>> The project contains two different flavours of the same test. One of
>>>>> them suffers from the problem, the other (due to my tests) not.
>>>>>>
>>>>>>
>>>>>> *** What does the testcase?
>>>>>> 1. Produces 1000 messages (100/s) and sends them to an "input" queue.
>>>>>> 2. Sends the messages from the "input" queue to an "output" queue.
>>>>>> 3. Finally consumes the messages from the "output" queue to count them.
>>>>>>
>>>>>>
>>>>>> *** What is the difference between the two test flavours?
>>>>>> - There is a "standard" flavour that suffers from the problem
>>>>>> - And there is a "noTxManager" flavour that seems to not have the
>>> problem
>>>>>> - The "standard" flavour is kind of a well known Camel/ActiveMQ
>>>>> configuration
>>>>>> - with a Spring transaction manager
>>>>>> - with a Spring transaction policy
>>>>>> - With a "transacted" flag in Camel routes
>>>>>> - The "noTxManager" flavour is a "simple" configuration
>>>>>> - no Spring transaction manager
>>>>>> - no Spring transaction policy
>>>>>> - no "transacted" flag in Camel routes
>>>>>> - BUT: "lazyCreateTransactionManager" = false (so routes are
>>>>> transacted too)
>>>>>>
>>>>>>
>>>>>> *** How to run the testcases?
>>>>>> 1. Replace "[yourBrokerHost]" with the hostname of your ActiveMQ broker
>>>>>> 2. Run the testcase as JUnit test
>>>>>> 3. When you see lots of console messages that messages are sent, stop
>>>>> your ActiveMQ broker (do not kill-9 it, just shut it down normally)
>>>>>> 4. Exceptions are thrown on the console output
>>>>>> 5. After some seconds start your broker again
>>>>>> 6. The test finish normally and after some seconds dumps a book keeping
>>>>> on the console
>>>>>>
>>>>>>
>>>>>> *** How to interpret the results?
>>>>>> - When the test is successful, no message is lost. You can run the test
>>>>> without broker shutdown/startup and it will obviously always be
>>> successful.
>>>>>> - When the test fails, one or more messages are lost between queue
>>>>> "input" and "output". In my tests I was not able to run the "standard"
>>>>> flavour three times in a row successfully. About every second run
>>> failed.
>>>>> In contrast, the "noTxManager" flavour never failed in my tests.
>>>>>>
>>>>>> The book keeping for a failed test looks like the following. In this
>>>>> example Message number 281 is arrived at the input queue but not at the
>>>>> output queue. So it is lost.
>>>>>>
>>>>>> Messages created by Client: 1000
>>>>>> Client Exceptions during send: 0 []
>>>>>>
>>>>>> Messages received at input queue: 993
>>>>>> Missing Messages at input queue: 7 [282,283,284,285,286,287,288]
>>>>>> Duplicate Messages at input queue: 0 []
>>>>>>
>>>>>> Messages received at output queue: 992
>>>>>> Missing Messages at output queue: 8
>>> [281,282,283,284,285,286,287,288]
>>>>>> Duplicate Messages at output queue: 0 []
>>>>>>
>>>>>> Lost Messages between Queues: 1 [281]
>>>>>>
>>>>>>
>>>>>> *** What is the problem?
>>>>>> A Redhat engineer tracked the problem down to a Spring JMS template
>>>>> behaviour that is kind of strange. If a Spring transaction manager is
>>>>> defined in the config, it will end up with two of them. Therefore the
>>> small
>>>>> time range where messages can get lost that arises only when you have a
>>>>> certain load.
>>>>>>
>>>>>>
>>>>>> *** So, what is my question?
>>>>>> - Does this really mean that it is unsafe to use the "standard" flavour
>>>>> of configuration?
>>>>>> - Is there another config with TxManager etc that works correctly?
>>>>>> - What are limits of the "noTxManager" config? When is it not
>>> sufficent?
>>>>>>
>>>>>> Regards
>>>>>> Stephan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> <CamelAmqTxTest.zip>
>>>>>
>>>>>
>>>
>>>
>
