[
https://issues.apache.org/jira/browse/NIFI-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11472:
------------------------------------
Priority: Minor (was: Major)
> Make PutFTP processor more multithread friendly
> -----------------------------------------------
>
> Key: NIFI-11472
> URL: https://issues.apache.org/jira/browse/NIFI-11472
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.21.0
> Environment: CentOS server, Java 11
> Reporter: Daniel Salwerowicz
> Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Problem happens when a PutFTP is set to run several concurrent tasks and two
> (or more ) FlowFiles come in and both need to create the same directory. One
> of them will create directory and succeed immediately while the other will
> try to create directory, but fail since it already exist, throw an error, the
> FlowFile will then be penalized and on second run will succeed.
> While it is not the biggest error, as files are getting transferred in the
> end, but the bulletins and errors are annoying, especially in production
> environment where you don't want to get unnecessary errors.
> We found that the solution involves a simple change to the
> {{FTPTransfer.java}} class in:
> {{nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FTPTransfer.java}}
> On line 398 and {{ensureDirectoryExists}} method you can simply add another
> if check which double checks that the directory exists when it fails to
> create one.
> {code:java}
> final boolean cdSuccessful = setWorkingDirectory(remoteDirectory);
> if (!cdSuccessful) {
> if (client.makeDirectory(remoteDirectory)) {
> logger.debug("Remote Directory not found: created directory [{}]",
> remoteDirectory);
> } else if (!setWorkingDirectory(remoteDirectory)) {
> // Double check that the dir exists as it might have been created in
> another thread
> throw new IOException("Failed to create remote directory " +
> remoteDirectory);
> }
> }{code}
> We have created a fix and will submit it as a Pull Request on GitHub
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
