steveloughran opened a new pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280
This changes directory tree deletion so that only files are incrementally
deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoked
This avoids directory tombstones being added above files/child directories,
which stop that subtree operation from actually finishing.
Also:
* callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* new statistic to measure #of objects deleted, alongside request count.
* callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
to clarify behavior.
Test enhancements in ITestDirectoryOperationCost, but I didn't manage to
replicate it there (didn't quite understand the cause); the FileContext
test works for regression testing.
Change-Id: I8f96536bc6ffaf548b3da1068b9bfc897d34a993
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
