Stephan Balmer created CAMEL-21056:
--------------------------------------
Summary: Delay when using readLockMinAge
Key: CAMEL-21056
URL: https://issues.apache.org/jira/browse/CAMEL-21056
Project: Camel
Issue Type: Bug
Components: camel-file
Affects Versions: 3.20.6
Reporter: Stephan Balmer
When using the file component with the readLockMinAge setting, consuming each
file incurs a delay the length of readLockMinAge . So when `readLockMinAge=5s`
and there are 1000 files, it takes 5000 seconds to consume all files.
Example route URL:
file:/tmp/test?readLock=changed&readLockMinAge=5s&recursive=true&delete=true
This has two disadvantages:
* The maximal rate at which files are processed is 1/readLockMinAge. This
severly limits throughput on a route.
* When a route is started it will initially wait for readLockMinAge before
processing a file. Even if the file was written long ago.
Camel up to version 3.20.5 processes files instantly as soon as the min age is
reached. Since version 3.20.6 files are not processed instantly even if they
are older than min age. Instead, the file component will wait for
readLockMinAge for each file individually.
This regression was introduced in commit
[https://github.com/apache/camel/commit/5459f1da2673205a6c24041a8631ac99088b5eae
|https://github.com/apache/camel/commit/5459f1da2673205a6c24041a8631ac99088b5eae]I
can sympathize with the goal of that change, as I've experienced that problem
as well because file-managers preserve last-changed dates when copying files to
new places. However I do judge the importance of throughput in the usual case
higher than avoiding premature processing of files in the edge case. I think
the change should be reverted. A separate strategy or option for that use-case
could be introduced.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
