[
https://issues.apache.org/jira/browse/NIFI-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513974#comment-17513974
]
Josef Zahner commented on NIFI-8483:
------------------------------------
We have upgraded to NiFi 1.15.3 and it seems that the issue is gone.
> Restart NiFi - duplicate SFTP flowfiles
> ---------------------------------------
>
> Key: NIFI-8483
> URL: https://issues.apache.org/jira/browse/NIFI-8483
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.13.2
> Environment: Java 1.8.0_282, CentOS 7, 8-Node Cluster
> Reporter: Josef Zahner
> Priority: Major
> Attachments: SFTP_failure.png
>
>
> Since the upgrade from NiFi 1.11.4 to 1.13.2 we faced an issue with the
> FetchSFTP & PutSFTP processors. We have a 8-Node NiFi cluster. Pattern is
> always ListSFTP (tracking timestamp) - FetchSFTP (and delete) and PutSFTP.
> If we do a restart of NiFi and NiFi comes back, we sometimes see flowfiles
> for FetchSFTP (not found) and PutSFTP (already present on disk) which have
> been processed successfully and have been stored already. So in fact we see
> flowfiles in a failure queue which have been save to disk with PutSFTP, which
> should never happen. The files are always small (a few MBs) and the network
> connectivity is insanely fast. The cluster shutdown is always before the
> grace period runs out. The attached screeshot shows an example where the
> FetchSFTP and the PutSFTP failure queue has files. Especially for the
> FetchSFTP this shouldn't be possible and if I do a restart with the command
> below, I would expect that within the grace period the processor has been
> stopped and it can't be processed twice.
> {code:java}
> /opt/nifi/bin/nifi.sh restart
> {code}
> At the moment we have no clue where the issue comes from and why it happens,
> so I can't provide an exact scenario to reproduce it. I only know that it
> sometimes happens after a restart of our 8-node cluster.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
