[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629032#comment-15629032
]
Steve Loughran commented on HADOOP-13650:
-----------------------------------------
Overall, like the code, but I do want the (proposed) s3 entry point to cover
all things, including s3guard. One port of call for s3 operations. This patch
could go in as an interim measure, with a separate JIRA "migrate to s3cmd"
being another subtask, one dependent on s3cmd existing.
h3. S3GuardTool
# L90: why not just switch on the string, have the default: clause raise the
exception. Same for Line 438
# Isn't the S3aFS going to create/init its store? So no need to do that again
in this class, just ask
for the (inited) one in S3AFS
# importDir should use listFiles(path, recursive=true) for scalability;
recursive directory walks die on S3.
There's one catch: listFiles only returns files, not directories. This could be
the time to implement one that returns the full tree, which we can start off in
s3a, then pull up into the full FS API later.
h3. MetadataStoreTestBase
Can the {{assertTrue() calls have meaningful strings to throw on failures?}}
h3. TestS3GuardCommand
* {{ContractTestUtils}} has util functions to create/touch files, etc. Not
worth importing for creating 1-byte files though, unless you want to use the
touch() call.
* {{testDiffCommand}} should include {{out.toString()}} in is assertion errors,
so that the failing logs get printed.
> S3Guard: Provide command line tools to manipulate metadata store.
> -----------------------------------------------------------------
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-13650-HADOOP-13345.000.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
