[jira] [Commented] (CAMEL-17648) Optimizing File component performance when filtering file names.
[ https://issues.apache.org/jira/browse/CAMEL-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502206#comment-17502206 ] Ja commented on CAMEL-17648: I created a small example patch for FileConsumer with a simple optimization that dramatically speeds up polling directory contents with exclude and include patterns. The only drawback I can see is problem with "recursive=true" witch requires editing existing include/exclude patterns to include required subdirectories. [^CAMEL-17648.patch] > Optimizing File component performance when filtering file names. > > > Key: CAMEL-17648 > URL: https://issues.apache.org/jira/browse/CAMEL-17648 > Project: Camel > Issue Type: Improvement > Components: camel-file >Reporter: Ja >Priority: Minor > Fix For: 3.x > > Attachments: CAMEL-17648.patch > > > Currently FileConsumer matches file names after creating GenericFile object > for each file witch can be time consuming especially in shared network > directories. > This could be optimized by matching on name based query parameters (fileName, > include, exclude, antExclude, antInclude, includeExt, excludeExt) before > creating GenericFile objects. > If the file name does not match creating GenericFile objects would not be > necessary. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CAMEL-17648) Optimizing File component performance when filtering file names.
[ https://issues.apache.org/jira/browse/CAMEL-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492463#comment-17492463 ] Ja commented on CAMEL-17648: With this optimalization in mind, also would be nice to have query parameter to limit directory size (the number of files contained in the directory) just after getting its content. So we could don’t even try polling hundreds of thousands of files. > Optimizing File component performance when filtering file names. > > > Key: CAMEL-17648 > URL: https://issues.apache.org/jira/browse/CAMEL-17648 > Project: Camel > Issue Type: Improvement > Components: camel-file >Reporter: Ja >Priority: Minor > Fix For: 3.x > > > Currently FileConsumer matches file names after creating GenericFile object > for each file witch can be time consuming especially in shared network > directories. > This could be optimized by matching on name based query parameters (fileName, > include, exclude, antExclude, antInclude, includeExt, excludeExt) before > creating GenericFile objects. > If the file name does not match creating GenericFile objects would not be > necessary. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CAMEL-17648) Optimizing File component performance when filtering file names.
[ https://issues.apache.org/jira/browse/CAMEL-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492447#comment-17492447 ] Claus Ibsen commented on CAMEL-17648: - Oh yeah we can likely do this kind of optimization when its filter only on the name - mind that the generic file component is also used by camel-ftp so changes in camel-file is also affecting this component. > Optimizing File component performance when filtering file names. > > > Key: CAMEL-17648 > URL: https://issues.apache.org/jira/browse/CAMEL-17648 > Project: Camel > Issue Type: Improvement > Components: camel-file >Reporter: Ja >Priority: Minor > > Currently FileConsumer matches file names after creating GenericFile object > for each file witch can be time consuming especially in shared network > directories. > This could be optimized by matching on name based query parameters (fileName, > include, exclude, antExclude, antInclude, includeExt, excludeExt) before > creating GenericFile objects. > If the file name does not match creating GenericFile objects would not be > necessary. -- This message was sent by Atlassian Jira (v8.20.1#820001)
