Sunil O created CAMEL-12839:
-------------------------------
Summary: Usage of readLockMinAge
Key: CAMEL-12839
URL: https://issues.apache.org/jira/browse/CAMEL-12839
Project: Camel
Issue Type: Improvement
Components: camel-core
Affects Versions: 2.17.0
Environment: We are using Camel 2.17.0.redhat-630187 in Linux (qa) and
windows (dev)
Reporter: Sunil O
We have a implementation scenario with File consumer polling large number of
folders (3000+) and 200K files per day with 1 mts SLA per file processing.
We also need to use readLock to avoid picking files which are being written.
In this scenario - we went for readLock=changed option. However this option
results in thread sleeping for minium time as specified by
readLockCheckInterval period option. While looking for workarounds - we found
the readLockMinAge option - which allows to pick up files which are old enough
without getting into the sleep mode. This has reduced the time for picking
files and thereby reducing the overall processing time.
However whenever a file is encountered with age below minage - the sleep occurs
as per the logic in FileChangedExclusiveReadLockStrategy. If this 'sleep' step
can be avoided when readLockMin age is specified - then insteading of sleeping
- the consumer can go on to pick other files. This modified behavior would be
useful in scenario where overall throughput and processing performance is
important than sequential processing etc.
So it would be good if one of the below is available
a) there is a separate ExclusiveReadLockStrategy similar to
FileChangedExclusiveReadLockStrategy which deals only with readLockMin age and
skips file if age is not met instead of sleeping.
Or
b) an option skip/sleep should be added for
FileChangedExclusiveReadLockStrategy when readLockMinAge is used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
