Steve Loughran created HADOOP-14757:

             Summary: S3AFileSystem.innerRename() to size metadatastore lists 
                 Key: HADOOP-14757
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: HADOOP-13345
            Reporter: Steve Loughran
            Priority: Minor
             Fix For: HADOOP-13345

In {{S3AFileSystem.innerRename()}}, various ArrayLists are created to track 
paths to update; these are created with the default size. It could/should be 
possible to allocate better, so avoid expensive array growth & copy operations 
while iterating through the list of entries.

# for a single file copy, sizes == 1
# for a recursive copy, the outcome of the first real LIST will either provide 
the actual size, or, if the list == the max response, a very large minimum size.

For #2, we'd need to get the hint of iterable length rather than just iterate 
through...some interface {{{IterableLength.expectedMinimumSize()}} could do 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to