Hi
You test case is kinda bogus as the route is triggered by a file
consumer, which splits the file and route from there.
So if the caller in this case is the file consumer, how do you suppose
the default error handler should "handle" this?
It does this by rolling back the file and try the file again on next poll.
So try with another kind of caller, for example try the dataset
endpoint in camel that can be used for sending in loads of messages
and verify they are routed as expected.
<route>
<from uri="file://inputdir?noop=true" />
<unmarshal>
<string />
</unmarshal>
<split>
<tokenize token="\n" />
<to uri="customDirect:messageRouter" />
</split>
</route>
On Mon, Sep 7, 2009 at 1:15 PM, thabach<[email protected]> wrote:
>
> I think we might suffer from some misunderstanding.
>
> - For the first segment, I do have a custom error handler configured to
> redeliver failed messages to the seda queues, we are on the same page.
>
> - In the second segment, in the DeadLetterChannelTest, I do use a
> DeadLetterChannel which does not, for some reason, deliver all messages.
> Whereas in the ExceptionBarrierTest, I have another custom error handler
> installed right after the seda queue, an ExceptionBarrier endpoint, which
> simply forwards erroneous exchanges to the deadletter queue and leaves no
> message behind. This second test does not use the DeadLetterChannel, but
> rather shall illustrate that the general setup is not flawed.
>
> In both test cases the DeadLetterChannel and the ExceptionBarrier
> respectively, are sitting right before the ExceptionThrower, which in both
> test cases deterministically throws 25 exceptional messages. Somehow in the
> DeadLetterChannelTest case, depending on the run only some 38 to 42 messages
> out of 45 make it to the deadletter queue.
>
> Christian.
>
>
>
> Claus Ibsen-2 wrote:
>>
>> On Mon, Sep 7, 2009 at 10:28 AM, thabach<[email protected]> wrote:
>>>
>>> Hi Claus
>>>
>>> thanks for having a look, I appreciate.
>>>
>>> I don't think it is a misconfiguration though. On the first segment to
>>> the
>>> seda queues I do want the default error handler to return the
>>> 'exception'al-exchanges (i.e. the exchanges suffering from a 'Queue full'
>>> exception), to the producer. Theses messages are not supposed to end up
>>> in
>>> the deadletter queue, but to be recovered by the producer
>>> (CustomDirectProducer in the original unit tests, posted) and resent to
>>> the
>>> seda queues.
>>>
>>
>> Ah okay that is fine. Then we are on same page now. If you do *not*
>> use the DeadLetterChannel then its your own responsible for handling
>> errors.
>> And whether you build you own custom component to do redelivery is your
>> choice.
>>
>>
>>
>>
>>> With these 'blocking' producer semantics installed, all messages make it
>>> to
>>> the second segment (i.e. the routes consuming from the seda queues). In
>>> the
>>> original unit tests the ExceptionThrower always throws on the expected 25
>>> messages, but not all of these 'exceptional' messages make it to the
>>> deadletter queue when using a DeadLetterChannel (DeadLetterChannelTest),
>>> whereas in the ExceptionBarrierTest - using our custom ExceptionBarrier
>>> component - they do.
>>>
>>> I am still under the impression of losing messages and don't get me
>>> wrong, I
>>> am happy to be told that it was just me (and my configuration) :) .
>>>
>>
>> No Camel does not lose messages. Its your own choice to do redelivery
>> and error handling in your own custom component.
>>
>>
>>> Cheers, Christian.
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> I had a look at it.
>>>> There are *no* issues in Camel. DeadLetterChannel does *not* loose
>>>> messages.
>>>>
>>>> The problem Christian has is that he has not properly configured error
>>>> handling in Camel.
>>>> He have mixed error handlers
>>>> - default error handler
>>>> - dead letter channel error handler
>>>>
>>>> And since when he sends to the SEDA queue from a producer that are
>>>> handled by the default error handler,
>>>> then his message is *not* moved to a dead letter queue, but returned
>>>> back to the caller as an exception. (as it used the default error
>>>> handler)
>>>>
>>>> Configuring Camel to use dead letter channel error handler for the
>>>> entire routes remedy Christians problem.
>>>>
>>>> I have added an unit test to Camel at:
>>>> http://svn.apache.org/viewvc?rev=811997&view=rev
>>>>
>>>>
>>>> On Fri, Sep 4, 2009 at 9:04 AM, Bach
>>>> Christian<[email protected]> wrote:
>>>>> Heya Camel Riders
>>>>>
>>>>> I stumbled upon a problem on having DeadLetterChannel doing the error
>>>>> handling in an staged architecture (all in-VM, non-distributed). Our
>>>>> batch processing application (uses Camel 2.0 and) is losing messages.
>>>>>
>>>>> It seems to only happen under peak loads, when SEDA queues become full
>>>>> and start to throw IllegalStateExceptions on performing the add()s to
>>>>> their internal BlockingQueues. Our producer recovers those exchanges
>>>>> and
>>>>> yield()s, before re-sending the Exchanges to the SEDA endpoint.
>>>>>
>>>>> I have put together two unit tests that reproducibly illustrate the
>>>>> problem and the absence thereof. Both unit tests are similar expect for
>>>>> the dead letter queue exception handling implementation used. Both have
>>>>> the SEDA queues, facing stages, configured to size=1 in order to
>>>>> emulate
>>>>> peak load semantics.
>>>>>
>>>>> The first test case (DeadLetterChannelTest) is configured to use
>>>>> Camel's
>>>>> DeadLetterChannel and reproducibly loses messages, depending on the run
>>>>> between 5 to 15 (out of 45) on my machine.
>>>>>
>>>>> The second test case (ExceptionBarrierTest) is configured to use a
>>>>> custom ExceptionBarrier endpoint in place of DeadLetterChannel. The
>>>>> ExceptionBarrierComponent is derived from a Camel DirectComponent with
>>>>> some custom code to handle Exchanges suffering from exceptions and
>>>>> forwarding the erroneous exchanges to the dead letter queue. With the
>>>>> ExceptionBarrier, which takes in its simplistic implementation
>>>>> advantage
>>>>> of our specific routes to some extent, no messages are lost.
>>>>>
>>>>> The unit test are attached and I'd be most interested in learning what
>>>>> is going wrong.
>>>>>
>>>>> Cheers, Christian.
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Lost-Messages-in-DeadLetterChannel.-tp25289238p25326799.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Lost-Messages-in-DeadLetterChannel.-tp25289238p25328848.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
