On Mon, Feb 22, 2010 at 1:01 PM, Claus Ibsen <[email protected]> wrote: > On Mon, Feb 22, 2010 at 12:57 PM, wmoussel <[email protected]> wrote: >> >> I considered decreasing the poll interval as well, but couldn't get cpu usage >> down enough for acceptable interval... >> >> I'm trying PollingConsumerPollStrategy but begin method is a void not a >> boolean, isn't it? >> > > Ah yeah it is. I can see we haven't made that a boolean to allow you > to deny polling. > You can create a ticket in JIRA so we can enhance this so you can do > that in Camel 2.3. >
I have created the ticket https://issues.apache.org/activemq/browse/CAMEL-2492 >> >> >> Claus Ibsen-2 wrote: >>> >>> On Mon, Feb 22, 2010 at 9:04 AM, wmoussel <[email protected]> wrote: >>>> >>>> Hi, >>>> >>>> Here's my initial issue. I have to poll for new files in a directory >>>> containing a lot of files (5000 to 20000) (and I can't move them after >>>> processing). >>>> >>>> When using idempotent option it takes way too much cpu (like 9% even when >>>> they aren't new files). >>>> >>> >>> How often are you gonna poll in those files? You can probably change >>> the delay to be less frequent. >>> >>> In any case somehow you gotta match all those 20000 files whether or >>> not you have processed them before. >>> >>> >>>> So i tried adding a filter so that the consumer would only take files >>>> less >>>> than 2 days old. It doesn't change much in the end. When I looked into >>>> how >>>> filter works in Java I got why it's still cpu consuming... >>>> >>>> So then I thought i would get recent file names with a unix command like >>>> find . -mtime -1 and use pollenrich to poll specific files. The way I >>>> understand how poll enrich works, it will poll the whole directory and >>>> then >>>> try to match filename afterwards. That is also cpu consuming since it >>>> gets >>>> all the files as exchanges prior to the match. Am I wrong ? >>>> >>> >>> pollEnrich is not suitable for this as its meant for polling and >>> aggregating 1 resource at a time. >>> >>>> So I'm basically trying to invoke a FileName to GenericFileExchange >>>> processor in the middle of my route. Is there a way to do this? >>>> >>>> I'm open to any suggestion. >>>> >>> >>> The file component supports using a custom PollingConsumerPollStrategy >>> http://camel.apache.org/polling-consumer.html >>> >>> You can then implement your own logic and return false in the being >>> method where there are no new files since last time. >>> >>> >>>> Thanks in advance :) >>>> >>>> Wandrille >>>> -- >>>> View this message in context: >>>> http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27683938.html >>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>> >>>> >>> >>> >>> >>> -- >>> Claus Ibsen >>> Apache Camel Committer >>> >>> Author of Camel in Action: http://www.manning.com/ibsen/ >>> Open Source Integration: http://fusesource.com >>> Blog: http://davsclaus.blogspot.com/ >>> Twitter: http://twitter.com/davsclaus >>> >>> >> >> -- >> View this message in context: >> http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27686162.html >> Sent from the Camel - Users mailing list archive at Nabble.com. >> >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Author of Camel in Action: http://www.manning.com/ibsen/ > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
