On Mon, Feb 22, 2010 at 9:04 AM, wmoussel <[email protected]> wrote: > > Hi, > > Here's my initial issue. I have to poll for new files in a directory > containing a lot of files (5000 to 20000) (and I can't move them after > processing). > > When using idempotent option it takes way too much cpu (like 9% even when > they aren't new files). >
How often are you gonna poll in those files? You can probably change the delay to be less frequent. In any case somehow you gotta match all those 20000 files whether or not you have processed them before. > So i tried adding a filter so that the consumer would only take files less > than 2 days old. It doesn't change much in the end. When I looked into how > filter works in Java I got why it's still cpu consuming... > > So then I thought i would get recent file names with a unix command like > find . -mtime -1 and use pollenrich to poll specific files. The way I > understand how poll enrich works, it will poll the whole directory and then > try to match filename afterwards. That is also cpu consuming since it gets > all the files as exchanges prior to the match. Am I wrong ? > pollEnrich is not suitable for this as its meant for polling and aggregating 1 resource at a time. > So I'm basically trying to invoke a FileName to GenericFileExchange > processor in the middle of my route. Is there a way to do this? > > I'm open to any suggestion. > The file component supports using a custom PollingConsumerPollStrategy http://camel.apache.org/polling-consumer.html You can then implement your own logic and return false in the being method where there are no new files since last time. > Thanks in advance :) > > Wandrille > -- > View this message in context: > http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27683938.html > Sent from the Camel - Users mailing list archive at Nabble.com. > > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
