Steve Loughran created HADOOP-17961:
---------------------------------------
Summary: s3 and abfs incremental listing: ask for a smaller first
batch
Key: HADOOP-17961
URL: https://issues.apache.org/jira/browse/HADOOP-17961
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/azure, fs/s3
Affects Versions: 3.3.2
Reporter: Steve Loughran
With code gradually adopting listStatusIncremental(), asking for a smaller
initial batch could permit faster ramp of result processing.
probably most significant on an s3 versioned bucket, as there the need to skip
tombstones can result in significantly slower listings -but could benefit ABFS
too
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
