[
https://issues.apache.org/jira/browse/NIFI-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-7216.
----------------------------
Resolution: Duplicate
> FetchSFTP can't delete or move files upon completion
> ----------------------------------------------------
>
> Key: NIFI-7216
> URL: https://issues.apache.org/jira/browse/NIFI-7216
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.10.0
> Reporter: Matt Rodriguez
> Assignee: Joe Witt
> Priority: Major
> Fix For: 1.12.0
>
>
> Possibly similar to NIFI-7177 as they're not able to delete files with the
> GetSFTP processor, but they're getting a different error than I am.
>
> I'm using the FetchSFTP processor to get data from a third party SFTP server,
> I don't have a lot of details on version or configuration on their end. I
> have noticed that neither the "Move File" or "Delete File" Completion
> Strategy options work.
> When using "Delete File" as the Completion Strategy I get no alert/bulletin
> from NiFi to show that anything went wrong, however, the original file has
> not been deleted. I can manually delete the file using an SFTP client with
> the same username/password that NiFi connects as.
> When using "Move File" as the Completion Strategy I get this warning message
> when NiFi tries to move the file after completion:
> {code:java}
> 2020-03-03 13:59:54,375 WARN [Timer-Driven Process Thread-2]
> o.a.nifi.processors.standard.FetchSFTP
> FetchSFTP[id=0344354b-3a49-316e-a571-adcca7b3e70e] Successfully fetched the
> content for
> StandardFlowFileRecord[uuid=dea58017-8f3f-4991-91a3-1ff8ca167b6c,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1583243711464-8, container=default,
> section=8], offset=4625, length=7672936],offset=0,name=TEST.csv,size=7672936]
> from ftp1.XXXXX.com:22/Test/TEST.csv but failed to rename the remote file due
> to java.io.FileNotFoundException: No such file or directory: {code}
> As with deletion, I am able manually move (rename) the file using an SFTP
> client with the same username/password that NiFi connects as.
> However, I will add that I'm only able to delete or move the file PRIOR to
> NiFi fetching the file. If I try to do it immediately after NiFi has fetched
> the file, I'll get the exact same "no such file or directory" error from the
> SFTP server when using my local client. If I wait some arbitrary amount of
> time after NiFi has fetched the file, I am then able to delete or move the
> file.
> I've also noticed the below sequence of events:
> # NiFi lists the files on the SFTP server
> # NiFi fetches the files on the SFTP server
> # NiFi attempts to delete or move the files on the SFTP server and fails
> # I immediately attempt to delete or move the files on the SFTP server and
> fail
> # I stop the FetchSFTP processor in NiFi
> # I immediately attempt to delete or move the files on the SFTP server and
> it succeeds
> This leads me to believe that there is some sort of locking behavior
> happening where the fetch operation keeps the file locked for some arbitrary
> amount of time or until the fetch processor is stopped, which is preventing
> any other operation from taking place on the file.
> Interestingly enough, I was able to get both the "Delete File" and "Move
> File" Completion Strategies to work when I spun up a basic SFTP server on my
> own. It sounds like there is something specific with how locks are handled
> with this third part SFTP server I'm connecting to and how NiFi uses (or
> re-uses) SFTP clients/connections.
> As mentioned above, I unfortunately don't have a lot of information about
> this SFTP server I'm connecting to, the vendor has been very tight-lipped
> about their configuration for some reason.
> If there's anything else you need me to provide, please let me know.
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
