Pierre Villard created NIFI-15685:
-------------------------------------
Summary: ListS3 V1 pagination fails when delimiter is not set
causing infinite loop for buckets with more than 1000 objects
Key: NIFI-15685
URL: https://issues.apache.org/jira/browse/NIFI-15685
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Pierre Villard
Assignee: Pierre Villard
In S3ObjectBucketLister.setNextMarker() (the V1 lister), the code
unconditionally used listObjectsResponse.nextMarker() as the pagination marker.
Per the AWS S3 ListObjects V1 API, nextMarker is only populated when a
delimiter is specified in the request. Without a delimiter, nextMarker()
returns null, which resets the listing back to the beginning of the bucket,
creating an infinite loop re-fetching the same first page of objects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
