Allow FileConsumer to check if a file is unchanged before processing it
-----------------------------------------------------------------------
Key: CAMEL-250
URL: https://issues.apache.org/activemq/browse/CAMEL-250
Project: Apache Camel
Issue Type: Improvement
Reporter: Aaron Crickenberger
Attachments: camel-core.unchangedFileConsumer.patch
The use case is a FileConsumer polling a directory that's populated by an FTP
server. Currently, the FileConsumer is almost guaranteed to grab files that
aren't complete. Setting the consumer delay to something large just reduces
the likelihood, and requiring external lock files or renaming (though
potentially more atomic) seems like too many moving parts.
In an attempt to combat this, I added an isUnchanged() method to FileConsumer
that is used similar to isMatched(), along with two properties, unchangedDelay
and unchangedSize. Setting unchangedDelay to eg: 5000 will cause FileConsumer
to avoid processing a file until its last modified time is at least 5 seconds
in the past. For large files being copied across a local filesystem, this
doesn't always work, because the modified date may be preserved even though the
file's not done copying. Thus, setting unchangedSize to true will cause
FileConsumer to avoid processing a file until its filesize remains unchanged
across two successive polls.
There's probably a more camel-like way to do this, maybe involving a polling
strategy, so that the technique could shared with RemoteFileConsumers, but that
made my brain hurt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
