Hi Tom I suggest to use a Wait Processor (to delay the fetch) together with DetectDuplicate Processor. In that way you will fetch the file only once and after it has been written completely (as long as you know how long it takes in max. to finish writing). I know it’s not nice but that’s how we do it for the moment… I’m waiting for this feature as well :-(.
Cheers Josef From: Arpad Boda <ab...@hortonworks.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Monday, 28 January 2019 at 10:17 To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: Minimum file age Hi, It’s on the way: https://issues.apache.org/jira/browse/NIFI-5977 :) Regards, Arpad From: Tomislav Novosel <to.novo...@gmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Monday, 28 January 2019 at 09:19 To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Minimum file age Hi all, I'm having issue with SFTPList processor in Nifi. When reading files from folder where another process writes files, it lists the same file multiple time and ingests file multiple times because modification date of file changes rapidly as another process writes to it. It appears that Nifi lists faster than external process writes, so before the end of writing (conversion of file from one format to another), Nifi lists file multiple time and then creates duplicates. There is no property of Minimum file age like in ListFile processor. How can I resolve this to wait for a moment when the file is converted completely and then to list file and pass it to FetchSFTP processor? Thanks in advance, Tom.
smime.p7s
Description: S/MIME Cryptographic Signature
