[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822564#comment-15822564
]
Aaron Fabbri commented on HADOOP-13650:
---------------------------------------
Thanks for the followup patch [~eddyxu]. +1 on the code review. I will try to
get some testing in this evening or early next week.
{quote}
AF> S3A's listFiles discovers non-empty directories
Thanks for catching this. The comments are outdated now. Since
LocatedFileStatus erased the isEmptyDir, the code here is still valid I think.
I modified the comments.
{quote}
Ah.. Another reason the isEmptyDirectory bit should probably be ignored by
MetadataStore. This will get addressed in HADOOP-13914, so we're good here.
{quote}
AF> Should we add to dirCache here?
dirCache is used in putParentsIfNotPresent(child); after this statement.
{quote}
Understood, you put the *parent* in the dirCache there. In this code though,
you are putting the "child" dir in MS, so you could also remember that the
child dir is already in MS. This current code might put the "child" dir in MS
two times (once here and again when you add its children), depending on
iteration order of listFiles(). This does not affect correctness (it is just
perf optimization), so I'm still +1 on this patch.
> S3Guard: Provide command line tools to manipulate metadata store.
> -----------------------------------------------------------------
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-13650-HADOOP-13345.000.patch,
> HADOOP-13650-HADOOP-13345.001.patch, HADOOP-13650-HADOOP-13345.002.patch,
> HADOOP-13650-HADOOP-13345.003.patch, HADOOP-13650-HADOOP-13345.004.patch,
> HADOOP-13650-HADOOP-13345.005.patch, HADOOP-13650-HADOOP-13345.006.patch,
> HADOOP-13650-HADOOP-13345.007.patch, HADOOP-13650-HADOOP-13345.008.patch,
> HADOOP-13650-HADOOP-13345.009.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
