[
https://issues.apache.org/activemq/browse/CAMEL-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51944#action_51944
]
Oliver Hecker commented on CAMEL-1650:
--------------------------------------
Yes, I see the problem. The way how it is implemented at the moment assures an
"at least once" quality of service.
Assuming that race conditions will not occur quite often this seems to be OK
for most cases.
I was testing this because I am looking for a way to manage scheduled jobs (via
Camel Quartz) in a clustered szenario
running the same camel config on multiple host in parallel. I am looking for a
way to assure that even if the job is triggered
on every host only one of them gets actually processed. Due to the timers
running more or less synchronous on all hosts
I am expecting that race conditions might occur quite often. (I actually also
investigated in using the cluster features of Quartz
itself but found it not working within Camel Quartz)
Removing the id in case of a failure will not work if there there is some
(technical) problem which prevents from deleting the entry
in the database. So if we crash after inserting in the database but before we
could deliver to the destination then the message then
even when retrying later the filter will filter out the message.
I was thinking of some intermediate state to be stored in the repository to
serve as a lock for the time between "contains" and "add".
But this requires quite some effort concerning coordination and detecting
failed instances in the cluster.
So the Interface of IdempotentRepository might be something like
isProcessedAndLockIfNot (the former "contains"):
- Returns true if the id is contained in state "processed"
- Returns false if id is not contained. Will insert id in state "locked"
- if id is contained in state "locked" then the call will be blocked until
- the id is either removed or goes to state processed, then proceed as
above or
- a timeout is reached which indicates that the process which set the
lock possibly crashed; in this case the lock will be stolen and false will be
returned
setProcessed (the former "add"):
- set state of ID to "processed"
unlock (new):
- delete id from the repository;
It might even be necessary to keep track of who aquired a lock to avoid messing
up if the locker tries to release a lock after the timeout occurred.
In any case this requires quite some interface and coding changes (and some
more thoughts about how to do it).
> Race condition in IdempotentConsumer
> ------------------------------------
>
> Key: CAMEL-1650
> URL: https://issues.apache.org/activemq/browse/CAMEL-1650
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Oliver Hecker
> Attachments: IdempotentConsumerTest.java
>
>
> A possible possible race condition exists in the IdempotentConsumer
> implementation:
> The code first checks in the MessageIdRepository if the message was already
> processed. If not then it processes the message and
> afterwards adds the id to the repository. (See also
> http://issues.apache.org/activemq/browse/CAMEL-1451). There is no locking
> between the check with "contains" and the insert with "add". So if multiple
> threads/instances try this in parallel for the same id, then
> it might happen that more than one finds the id not yet contained in the
> repository and the same message is processed multiple
> times.
> I enclose an extended version of IdempotentConsumerTest which illustrates the
> problem.
> It is important to note that even if the test demonstrates the issue with an
> MemoryIdempotentRepository a solution should also
> address the case of a database based respository in a clustered environment.
> So this might imply that some locking mechanism on the
> database is required.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
