This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch camel-2.20.x
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/camel-2.20.x by this push:
     new faf5598  CAMEL-12337: camel-ftp - ignoreFileNotFoundOrPermissionError 
should also ignore dir not found or list dir permission error.
faf5598 is described below

commit faf5598edc5bba66c532ce84ce412c20c545d16c
Author: Claus Ibsen <claus.ib...@gmail.com>
AuthorDate: Sat Mar 10 10:40:32 2018 +0100

    CAMEL-12337: camel-ftp - ignoreFileNotFoundOrPermissionError should also 
ignore dir not found or list dir permission error.
---
 components/camel-ftp/src/main/docs/ftp-component.adoc |  4 ++--
 .../camel-ftp/src/main/docs/ftps-component.adoc       |  4 ++--
 .../camel-ftp/src/main/docs/sftp-component.adoc       |  4 ++--
 .../camel/component/file/remote/FtpConsumer.java      | 16 ++++++++++++----
 .../file/remote/RemoteFileConfiguration.java          |  4 ++--
 .../camel/component/file/remote/SftpConsumer.java     | 19 ++++++++++++++-----
 6 files changed, 34 insertions(+), 17 deletions(-)

diff --git a/components/camel-ftp/src/main/docs/ftp-component.adoc 
b/components/camel-ftp/src/main/docs/ftp-component.adoc
index 2224e7f..c015bbb 100644
--- a/components/camel-ftp/src/main/docs/ftp-component.adoc
+++ b/components/camel-ftp/src/main/docs/ftp-component.adoc
@@ -123,7 +123,7 @@ with the following path and query parameters:
 | *exceptionHandler* (consumer) | To let the consumer use a custom 
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this 
options is not in use. By default the consumer will deal with exceptions that 
will be logged at WARN or ERROR level and ignored. |  | ExceptionHandler
 | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer 
creates an exchange. |  | ExchangePattern
 | *handleDirectoryParser AbsoluteResult* (consumer) | Allows you to set how 
the consumer will handle subfolders and files in the path if the directory 
parser results in with absolute paths The reason for this is that some FTP 
servers may return file names with absolute paths and if so then the FTP 
component needs to handle this by converting the returned path into a relative 
path. | false | boolean
-| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
trying to download a file which does not exist or due to permission error. By 
default when a file does not exists or insufficient permission then an 
exception is thrown. Setting this option to true allows to ignore that instead. 
| false | boolean
+| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
(trying to list files in directories or when downloading a file) which does not 
exist or due to permission error. By default when a directory or file does not 
exists or insufficient permission then an exception is thrown. Setting this 
option to true allows to ignore that instead. | false | boolean
 | *inProgressRepository* (consumer) | A pluggable in-progress repository 
org.apache.camel.spi.IdempotentRepository. The in-progress repository is used 
to account the current in progress files being consumed. By default a memory 
based repository is used. |  | String>
 | *localWorkDirectory* (consumer) | When consuming a local work directory can 
be used to store the remote file content directly in local files to avoid 
loading the content into memory. This is beneficial if you consume a very big 
remote file and thus can conserve memory. |  | String
 | *onCompletionException Handler* (consumer) | To use a custom 
org.apache.camel.spi.ExceptionHandler to handle any thrown exceptions that 
happens during the file on completion process where the consumer does either a 
commit or rollback. The default implementation will log any exception at WARN 
level and ignore. |  | ExceptionHandler
@@ -160,7 +160,7 @@ with the following path and query parameters:
 | *throwExceptionOnConnect Failed* (advanced) | Should an exception be thrown 
if connection failed (exhausted) By default exception is not thrown and a WARN 
is logged. You can use this to enable exception being thrown and handle the 
thrown exception from the org.apache.camel.spi.PollingConsumerPollStrategy 
rollback method. | false | boolean
 | *timeout* (advanced) | Sets the data timeout for waiting for reply Used only 
by FTPClient | 30000 | int
 | *antExclude* (filter) | Ant style filter exclusion. If both antInclude and 
antExclude are used antExclude takes precedence over antInclude. Multiple 
exclusions may be specified in comma-delimited format. |  | String
-| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant fiter | 
true | boolean
+| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant filter | 
true | boolean
 | *antInclude* (filter) | Ant style filter inclusion. Multiple inclusions may 
be specified in comma-delimited format. |  | String
 | *eagerMaxMessagesPerPoll* (filter) | Allows for controlling whether the 
limit from maxMessagesPerPoll is eager or not. If eager then the limit is 
during the scanning of files. Where as false would scan all files and then 
perform sorting. Setting this option to false allows for sorting all files 
first and then limit the poll. Mind that this requires a higher memory usage as 
all file details are in memory to perform the sorting. | true | boolean
 | *exclude* (filter) | Is used to exclude files if filename matches the regex 
pattern (matching is case in-senstive). Notice if you use symbols such as plus 
sign and others you would need to configure this using the RAW() syntax if 
configuring this as an endpoint uri. See more details at configuring endpoint 
uris |  | String
diff --git a/components/camel-ftp/src/main/docs/ftps-component.adoc 
b/components/camel-ftp/src/main/docs/ftps-component.adoc
index 60ed2f7..a3d42eb 100644
--- a/components/camel-ftp/src/main/docs/ftps-component.adoc
+++ b/components/camel-ftp/src/main/docs/ftps-component.adoc
@@ -83,7 +83,7 @@ with the following path and query parameters:
 | *exceptionHandler* (consumer) | To let the consumer use a custom 
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this 
options is not in use. By default the consumer will deal with exceptions that 
will be logged at WARN or ERROR level and ignored. |  | ExceptionHandler
 | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer 
creates an exchange. |  | ExchangePattern
 | *handleDirectoryParser AbsoluteResult* (consumer) | Allows you to set how 
the consumer will handle subfolders and files in the path if the directory 
parser results in with absolute paths The reason for this is that some FTP 
servers may return file names with absolute paths and if so then the FTP 
component needs to handle this by converting the returned path into a relative 
path. | false | boolean
-| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
trying to download a file which does not exist or due to permission error. By 
default when a file does not exists or insufficient permission then an 
exception is thrown. Setting this option to true allows to ignore that instead. 
| false | boolean
+| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
(trying to list files in directories or when downloading a file) which does not 
exist or due to permission error. By default when a directory or file does not 
exists or insufficient permission then an exception is thrown. Setting this 
option to true allows to ignore that instead. | false | boolean
 | *inProgressRepository* (consumer) | A pluggable in-progress repository 
org.apache.camel.spi.IdempotentRepository. The in-progress repository is used 
to account the current in progress files being consumed. By default a memory 
based repository is used. |  | String>
 | *localWorkDirectory* (consumer) | When consuming a local work directory can 
be used to store the remote file content directly in local files to avoid 
loading the content into memory. This is beneficial if you consume a very big 
remote file and thus can conserve memory. |  | String
 | *onCompletionException Handler* (consumer) | To use a custom 
org.apache.camel.spi.ExceptionHandler to handle any thrown exceptions that 
happens during the file on completion process where the consumer does either a 
commit or rollback. The default implementation will log any exception at WARN 
level and ignore. |  | ExceptionHandler
@@ -120,7 +120,7 @@ with the following path and query parameters:
 | *throwExceptionOnConnect Failed* (advanced) | Should an exception be thrown 
if connection failed (exhausted) By default exception is not thrown and a WARN 
is logged. You can use this to enable exception being thrown and handle the 
thrown exception from the org.apache.camel.spi.PollingConsumerPollStrategy 
rollback method. | false | boolean
 | *timeout* (advanced) | Sets the data timeout for waiting for reply Used only 
by FTPClient | 30000 | int
 | *antExclude* (filter) | Ant style filter exclusion. If both antInclude and 
antExclude are used antExclude takes precedence over antInclude. Multiple 
exclusions may be specified in comma-delimited format. |  | String
-| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant fiter | 
true | boolean
+| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant filter | 
true | boolean
 | *antInclude* (filter) | Ant style filter inclusion. Multiple inclusions may 
be specified in comma-delimited format. |  | String
 | *eagerMaxMessagesPerPoll* (filter) | Allows for controlling whether the 
limit from maxMessagesPerPoll is eager or not. If eager then the limit is 
during the scanning of files. Where as false would scan all files and then 
perform sorting. Setting this option to false allows for sorting all files 
first and then limit the poll. Mind that this requires a higher memory usage as 
all file details are in memory to perform the sorting. | true | boolean
 | *exclude* (filter) | Is used to exclude files if filename matches the regex 
pattern (matching is case in-senstive). Notice if you use symbols such as plus 
sign and others you would need to configure this using the RAW() syntax if 
configuring this as an endpoint uri. See more details at configuring endpoint 
uris |  | String
diff --git a/components/camel-ftp/src/main/docs/sftp-component.adoc 
b/components/camel-ftp/src/main/docs/sftp-component.adoc
index 398dc1c..4e1598a 100644
--- a/components/camel-ftp/src/main/docs/sftp-component.adoc
+++ b/components/camel-ftp/src/main/docs/sftp-component.adoc
@@ -72,7 +72,7 @@ with the following path and query parameters:
 | *download* (consumer) | Whether the FTP consumer should download the file. 
If this option is set to false then the message body will be null but the 
consumer will still trigger a Camel Exchange that has details about the file 
such as file name file size etc. It's just that the file will not be 
downloaded. | false | boolean
 | *exceptionHandler* (consumer) | To let the consumer use a custom 
ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this 
options is not in use. By default the consumer will deal with exceptions that 
will be logged at WARN or ERROR level and ignored. |  | ExceptionHandler
 | *exchangePattern* (consumer) | Sets the exchange pattern when the consumer 
creates an exchange. |  | ExchangePattern
-| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
trying to download a file which does not exist or due to permission error. By 
default when a file does not exists or insufficient permission then an 
exception is thrown. Setting this option to true allows to ignore that instead. 
| false | boolean
+| *ignoreFileNotFoundOr PermissionError* (consumer) | Whether to ignore when 
(trying to list files in directories or when downloading a file) which does not 
exist or due to permission error. By default when a directory or file does not 
exists or insufficient permission then an exception is thrown. Setting this 
option to true allows to ignore that instead. | false | boolean
 | *inProgressRepository* (consumer) | A pluggable in-progress repository 
org.apache.camel.spi.IdempotentRepository. The in-progress repository is used 
to account the current in progress files being consumed. By default a memory 
based repository is used. |  | String>
 | *localWorkDirectory* (consumer) | When consuming a local work directory can 
be used to store the remote file content directly in local files to avoid 
loading the content into memory. This is beneficial if you consume a very big 
remote file and thus can conserve memory. |  | String
 | *onCompletionException Handler* (consumer) | To use a custom 
org.apache.camel.spi.ExceptionHandler to handle any thrown exceptions that 
happens during the file on completion process where the consumer does either a 
commit or rollback. The default implementation will log any exception at WARN 
level and ignore. |  | ExceptionHandler
@@ -106,7 +106,7 @@ with the following path and query parameters:
 | *throwExceptionOnConnect Failed* (advanced) | Should an exception be thrown 
if connection failed (exhausted) By default exception is not thrown and a WARN 
is logged. You can use this to enable exception being thrown and handle the 
thrown exception from the org.apache.camel.spi.PollingConsumerPollStrategy 
rollback method. | false | boolean
 | *timeout* (advanced) | Sets the data timeout for waiting for reply Used only 
by FTPClient | 30000 | int
 | *antExclude* (filter) | Ant style filter exclusion. If both antInclude and 
antExclude are used antExclude takes precedence over antInclude. Multiple 
exclusions may be specified in comma-delimited format. |  | String
-| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant fiter | 
true | boolean
+| *antFilterCaseSensitive* (filter) | Sets case sensitive flag on ant filter | 
true | boolean
 | *antInclude* (filter) | Ant style filter inclusion. Multiple inclusions may 
be specified in comma-delimited format. |  | String
 | *eagerMaxMessagesPerPoll* (filter) | Allows for controlling whether the 
limit from maxMessagesPerPoll is eager or not. If eager then the limit is 
during the scanning of files. Where as false would scan all files and then 
perform sorting. Setting this option to false allows for sorting all files 
first and then limit the poll. Mind that this requires a higher memory usage as 
all file details are in memory to perform the sorting. | true | boolean
 | *exclude* (filter) | Is used to exclude files if filename matches the regex 
pattern (matching is case in-senstive). Notice if you use symbols such as plus 
sign and others you would need to configure this using the RAW() syntax if 
configuring this as an endpoint uri. See more details at configuring endpoint 
uris |  | String
diff --git 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
index 1d72438..af2cd25 100644
--- 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
+++ 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpConsumer.java
@@ -117,10 +117,18 @@ public class FtpConsumer extends 
RemoteFileConsumer<FTPFile> {
         log.trace("Polling directory: {}", dir);
         List<FTPFile> files = null;
         if (isUseList()) {
-            if (isStepwise()) {
-                files = operations.listFiles();
-            } else {
-                files = operations.listFiles(dir);
+            try {
+                if (isStepwise()) {
+                    files = operations.listFiles();
+                } else {
+                    files = operations.listFiles(dir);
+                }
+            } catch (GenericFileOperationFailedException e) {
+                if (ignoreCannotRetrieveFile(null, null, e)) {
+                    log.debug("Cannot list files in directory {} due directory 
does not exists or file permission error.", dir);
+                } else {
+                    throw e;
+                }
             }
         } else {
             // we cannot use the LIST command(s) so we can only poll a named 
file
diff --git 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/RemoteFileConfiguration.java
 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/RemoteFileConfiguration.java
index b85a6ec..eb9e4f5 100644
--- 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/RemoteFileConfiguration.java
+++ 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/RemoteFileConfiguration.java
@@ -366,9 +366,9 @@ public abstract class RemoteFileConfiguration extends 
GenericFileConfiguration {
     }
 
     /**
-     * Whether to ignore when trying to download a file which does not exist 
or due to permission error.
+     * Whether to ignore when (trying to list files in directories or when 
downloading a file), which does not exist or due to permission error.
      * <p/>
-     * By default when a file does not exists or insufficient permission, then 
an exception is thrown.
+     * By default when a directory or file does not exists or insufficient 
permission, then an exception is thrown.
      * Setting this option to <tt>true</tt> allows to ignore that instead.
      */
     public void setIgnoreFileNotFoundOrPermissionError(boolean 
ignoreFileNotFoundOrPermissionError) {
diff --git 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpConsumer.java
 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpConsumer.java
index 048c5ac..f534016 100644
--- 
a/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpConsumer.java
+++ 
b/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpConsumer.java
@@ -111,12 +111,21 @@ public class SftpConsumer extends 
RemoteFileConsumer<ChannelSftp.LsEntry> {
         }
 
         log.trace("Polling directory: {}", dir);
-        List<ChannelSftp.LsEntry> files;
-        if (isStepwise()) {
-            files = operations.listFiles();
-        } else {
-            files = operations.listFiles(dir);
+        List<ChannelSftp.LsEntry> files = null;
+        try {
+            if (isStepwise()) {
+                files = operations.listFiles();
+            } else {
+                files = operations.listFiles(dir);
+            }
+        } catch (GenericFileOperationFailedException e) {
+            if (ignoreCannotRetrieveFile(null, null, e)) {
+                log.debug("Cannot list files in directory {} due directory 
does not exists or file permission error.", dir);
+            } else {
+                throw e;
+            }
         }
+
         if (files == null || files.isEmpty()) {
             // no files in this directory to poll
             log.trace("No files found in directory: {}", dir);

-- 
To stop receiving notification emails like this one, please contact
davscl...@apache.org.

Reply via email to