[
https://issues.apache.org/jira/browse/HADOOP-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812647#comment-15812647
]
Mingliang Liu commented on HADOOP-13960:
----------------------------------------
Thanks [~eddyxu] for discussion, review and commit!
Aside from the unit test against DynamoDBLocal simulator, I should also have
posted the integration test results as following (region US-West-1):
{code}
$ mvn -Dit.test='ITestS3A*' -Dscale -q clean verify
Results :
Tests in error:
ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » IO Failed to
instan...
ITestS3ACredentialsInURL.testInstantiateFromURL:86 » IO Failed to instantiate
...
ITestS3ACredentialsInURL.testInvalidCredentialsFail:127 » AccessDenied
s3a://m...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:669->FileSystemContractBaseTest.rename:525
» AWSServiceIO
Tests run: 321, Failures: 0, Errors: 4, Skipped: 8
{code}
According to our recent discussion, this is a good run (failures are tracked
elsewhere).
> Initialize DynamoDBMetadataStore without associated S3AFileSystem
> -----------------------------------------------------------------
>
> Key: HADOOP-13960
> URL: https://issues.apache.org/jira/browse/HADOOP-13960
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-13960-HADOOP-13345.000.patch
>
>
> Per the discussion in [HADOOP-13650], it's helpful to initialize a
> DynamoDBMetadataStore object without S3AFileSystem. In the current code, we
> can achieve this via {{DynamoDBMetadataStore#initialize(Configuration)}}.
> However, users still have to provide the associated S3AFileSystem URI in the
> configuration, by means of either setting the {{fs.defaultFS}} in
> configuration file or {{-fs s3://bucket}} command line parameter. Setting the
> default FileSystem in configuration seems not necessary as the command line
> is to manipulate metadata store, e.g. command line tools on an existing HDFS
> cluster.
> This JIRA is to track the effort of initializing a DynamoDBMetadataStore
> without associating any S3 buckets, so that S3AFileSystem is not needed.
> Users have to specify in configuration the DynamoDB endpoints (for region)
> and table name along with credentials, AWS client settings.
> See [~eddyxu] and [~liuml07]'s comments in [HADOOP-13650] for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
