Jon,
How about if we enhance the file consumer to keep track of files that
have already been processed instead of using a timestamp? The timestamp
approach is a bit error-prone (just touching the file by accident can
set it off again).
If we provide multiple implementations for the storage mechanism to keep
this information, we can cover a lot of use cases (similar to the
message id store for an idempotent consumer):
- an in-memory store for testing purposes
- a file-based implementation for basic production environments
- a database- or ldap-backed implementation for clustered environments,
where a file can arrive through multiple directories
Regards,
Gert
Jon Anstey schreef:
The algorithm that checks whether a file should be consumed based on
timestamp has been deprecated for a while now (see
http://activemq.apache.org/camel/file.html). I've removed this on my local
branch only to realize that it introduces a bit of an ugly problem...
essentially since files will be processed always (modified or not) in the
case of noop=true or if a fault has been set, the same file will be
processed over and over again... not good!
The original intent of removing the timestamp checking was to simplify the
consumer. I think that in trying to get around this new issue we may make it
even more complicated!
I'm wondering if there is a simple solution to this that I'm just not seeing
yet or if maybe this issue was discussed before...
