[
https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866605#comment-16866605
]
Steve Loughran commented on HADOOP-16380:
-----------------------------------------
Note, there are some other places where innerGetFileStatus(path, true) are used
which are vulnerable to the same problem
* innerRename checks the empty dir status on src and dest paths. the source
check doesn't seem to need it, but the destDir check is needed to block
renaming onto a non-empty directory
* Some tests call it too; less important
This problem only exists when S3Guard is running (it has the tombstones) , and
then iff the deleted paths come first in the listing. i.e. it doesn't surface
if the file is {{aaa}} and the dir {{zzz}}, but it does the other way around
One possible fix related to HADOOP-15988: even if !authoritative (remember, /
is always tagged as nonauth), then we could still set the emptyDir marker ==
false if we had 1+ non-expired child. As we could be confident that yes, it was
not empty. That would be enough to block the two operations (rename, delete)
which care not that a directory has children, only that it is non-empty. Does
that make sense?
+[~gabor.bota], [~ajfabbri]
> ITestS3AContractRootDir failing on trunk: tombstones mislead about directory
> empty status
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-16380
> URL: https://issues.apache.org/jira/browse/HADOOP-16380
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Critical
>
> I'm seeing reproducible failures of {{ITestS3AContractRootDir}} which look
> like consistency problems *even when S3Guard is enabled*.
> Suspicion: root dir listings are still inconsistent, due to the way we don't
> keep root entries in the table
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
