[
https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886270#comment-16886270
]
Gabor Bota commented on HADOOP-16380:
-------------------------------------
The problem seems a little more complex. The test that reproduces the issue
uses an OOB operation, so it will be classified as an OOB test.
* OOB: we should recover if there's a tombstone hiding the listing of a
directory which was deleted with MS and re-created as an OOB op.
* Root cause for this is not happening: we don't filter out results in
MetadartaStore#listChildren generally in any implementation. We should filter
expired results, so expired tombstones and entries won't get into the listing.
Other issues with deleting the root:
* In ABFS we don't let the clients run the {{delete /}} operation if it's not
recursive. We should do the same for S3A, so I'll add this behavior with this
jira, and create another for those two other I described earlier.
> S3Guard tombstones can mislead about root directory empty status
> ----------------------------------------------------------------
>
> Key: HADOOP-16380
> URL: https://issues.apache.org/jira/browse/HADOOP-16380
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.2.0, 3.0.3, 3.3.0, 3.1.2
> Reporter: Steve Loughran
> Assignee: Gabor Bota
> Priority: Major
>
> If S3AFileSystem does an S3 LIST restricted to a single object to see if a
> directory is empty, and the single entry found has a tombstone marker (either
> from an inconsistent DDB Table or from an eventually consistent LIST) then it
> will consider the directory empty, _even if there is 1+ entry which is not
> deleted_
> We need to make sure the calculation of whether a directory is empty or not
> is resilient to this, efficiently.
> It surfaces as an issue two places
> * delete(path) (where it may make things worse)
> * rename(src, dest), where a check is made for dest != an empty directory.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
