[
https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883053#comment-16883053
]
Steve Loughran commented on HADOOP-16380:
-----------------------------------------
Downgraded from critical to major, as it only seems to affect the root
directory.
We only check for a dir being empty in three places
* delete dir
* rename (check the dest)
* delete / -where we decided whether to no-op ( empty dir) or reject
(non-empty-dir)
Now, if you look at filesystem.md we don't come up with a good answer of what
to do on an rm /
We could just have it return false on an attempt to delete a root dir, empty or
not, which is essentially what abfs does. At which point, no need to even check
for the dir existing,
yes, its a subtle change in how s3a:// handles rm /, but I think it's semantics
was based on what s3n did, rather than a notion of what is the "one true way".
After all, we explicitly don't allow an rm -rf / to either delete the root, or
deleted the bucket entirely
We could u
> S3Guard tombstones can mislead about root directory empty status
> ----------------------------------------------------------------
>
> Key: HADOOP-16380
> URL: https://issues.apache.org/jira/browse/HADOOP-16380
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.2.0, 3.0.3, 3.3.0, 3.1.2
> Reporter: Steve Loughran
> Assignee: Gabor Bota
> Priority: Major
>
> If S3AFileSystem does an S3 LIST restricted to a single object to see if a
> directory is empty, and the single entry found has a tombstone marker (either
> from an inconsistent DDB Table or from an eventually consistent LIST) then it
> will consider the directory empty, _even if there is 1+ entry which is not
> deleted_
> We need to make sure the calculation of whether a directory is empty or not
> is resilient to this, efficiently.
> It surfaces as an issue two places
> * delete(path) (where it may make things worse)
> * rename(src, dest), where a check is made for dest != an empty directory.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
