CHANDAN KUMAR created NIFI-14326:
------------------------------------
Summary: ListSFTP :Remote Poll Batch : does not list files beyond
configured value
Key: NIFI-14326
URL: https://issues.apache.org/jira/browse/NIFI-14326
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 2.2.0, 1.28.1, 1.26.0
Reporter: CHANDAN KUMAR
Test case :
=========
sftp location which contains close to 100 files of different timestamp
Run ListSFTP with *Remote Poll Batch Size* to 10
ListSFTP is set run with *Tracking Timestamps*
*ListSFTP just listed 10 files in one go and after that it is not listing
anything*
if I look at the state of ListSFTP it has stored the file which is Not most the
recent one.
Further I keep creating New few files on SFTP but No changes on listing
it Is not listing anything beyond those 10 files.
I did stopped and started back No luck
tried clearing state and started back again same 10 files
As per document *Remote Poll Batch Size*
_The value specifies how many file paths to find in a given directory on the
remote system when doing a file listing. This value in general should not need
to be modified but when polling against a remote system with a tremendous
number of files this value can be critical. Setting this value too high can
result very poor performance and setting it too low can cause the flow to be
slower than normal._
I though it would keep listing new files with each run a batch of 10 files and
it will finish listing all of them but that is not the case.
Seems to be Bug
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
