Hi guys We need your advice,… we use the ListSFTP processor to read files on a remote folder. The files gets written like that:
* File1 * File2 * File3 (at the time of “ls” command this file is growing, we don’t know how big it gets or when writing is finished) So the application on that remote folder writes the files one by one, always one file grows until a certain size (not predictable). So our issue is that NiFi reads the same file multiple time as it grows and the modify timestamp changes. But as we don’t know when writing is finished we don’t know how to drop all “incomplete” ListSFTP flowfiles of the “File3” in the example. Any help in this, in theory, pretty simple case? The best solution would be to tell the ListSFTP that it should ignore files with a timestamp of lastmodified from at least about x minutes. But there is no such option. We need somehow a ListSFTP delay and we shouldn’t read too new files. But we have no idea how to do that. Greetings Josef
smime.p7s
Description: S/MIME Cryptographic Signature
