[jira] [Commented] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818972#comment-17818972 ] ASF subversion and git services commented on NIFI-12772: Commit 89b618cc052a19bb4bcdbe8f7067ad996aad614d in nifi's branch refs/heads/main from Tom Brisland [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=89b618cc05 ] NIFI-12772 Expose REMOTE_POLL_BATCH_SIZE property for ListSFTP This closes #8390. revert max results optimisation + unnecessary import Signed-off-by: Joseph Witt > Remote poll batch size not exposed for ListSFTP > --- > > Key: NIFI-12772 > URL: https://issues.apache.org/jira/browse/NIFI-12772 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Tom Brisland >Priority: Minor > Time Spent: 1h 40m > Remaining Estimate: 0h > > I was planning on adding support for a batch size in ListSFTP due to some > issues we're seeing with a large number of files on an SFTP server. > Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all > the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816246#comment-17816246 ] Tom Brisland commented on NIFI-12772: - I'll do some manual testing tomorrow but I did think from an initial browse that it was already implemented > Remote poll batch size not exposed for ListSFTP > --- > > Key: NIFI-12772 > URL: https://issues.apache.org/jira/browse/NIFI-12772 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Tom Brisland >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > I was planning on adding support for a batch size in ListSFTP due to some > issues we're seeing with a large number of files on an SFTP server. > Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all > the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12772) Remote poll batch size not exposed for ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816245#comment-17816245 ] Joe Witt commented on NIFI-12772: - The challenge here is whether the underlying library makes it possible to control that. If it does then great. If not then...not sure how to solve. A similar difficulty existed back in the day for ListFile or equivalent cases. Java itself didnt expose a way to control this. You could bring down a system just by making a listing call in a large directory. New IO mechanisms resolved that scenario but SFTP is a different animal. Love the idea just need to make sure the library exposes that power. > Remote poll batch size not exposed for ListSFTP > --- > > Key: NIFI-12772 > URL: https://issues.apache.org/jira/browse/NIFI-12772 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Tom Brisland >Priority: Minor > > I was planning on adding support for a batch size in ListSFTP due to some > issues we're seeing with a large number of files on an SFTP server. > Thankfully, it seems that REMOTE_POLL_BATCH_SIZE is already supported in all > the FTP processor variations, just not exposed in ListSFTP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
