Hi Yeah you can use idempotent repo - and then a regular changed read lock to avoid processing the file if its currently being written
On Mon, Nov 23, 2020 at 9:58 AM Robin Vanderhallen <[email protected]> wrote: > > Hi Claus, > > Thanks for the reply. Can I still use the idempotent repository to prevent > different nodes from processing the same file even though > the idempotent read lock is not supported? I don’t see any limitations on > using an idempotent repository on the FTP endpoint. > On 23 Nov 2020, 09:44 +0100, Claus Ibsen <[email protected]>, wrote: > > The ftp component does not support idempotent read locks, so you > > cannot do what you do. > > > > On Sun, Nov 22, 2020 at 8:05 AM Robin Vanderhallen > > <[email protected]> wrote: > > > > > > Hi, > > > > > > I’m using file and FTP consumer endpoints (Camel 2.22.1) in a clustered > > > setup, backed with Hazelcast and write-through persistence using the > > > following endpoint options: > > > idempotent=true&idempotentRepository=#hazelcastIdempotentRepository&idempotentKey=#keyExpression&readLock=idempotent-changed > > > for the file endpoint > > > idempotent=true&idempotentRepository=#hazelcastIdempotentRepository&idempotentKey=#keyExpression&readLock=changed > > > for the FTP endpoint > > > > > > I discovered that there is a difference in the behaviour of both > > > endpoints: > > > > > > file component: confirm method is invoked after processing the files > > > FTP component: confirm method is not invoked after processing the files > > > > > > When readLock=idempotent-changed is set on the file endpoint, > > > FileIdempotentChangedRepositoryReadLockStrategy is being used and the > > > releaseExclusiveReadLockOnCommit method is > > > invoked which triggers the confirm method on the idempotent repository. > > > According to the Camel docs this type of read lock is not supported by > > > the FTP component. In my setup FtpChangedExclusiveReadLockStrategy is > > > being used, which is never invoking the confirm method > > > on the idempotent repository (releaseExclusiveReadLockOnCommit is an > > > empty method). > > > > > > I read the following in the Camel in Action book: To commit the Exchange, > > > the confirm method is invoked on the idempotent repository > > > > > > Does this imply that the Exchange is never committed in case of the FTP > > > component. Is this expected behaviour? > > > > > > Thanks, > > > > > > Robin > > > > > > > > -- > > Claus Ibsen > > ----------------- > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
