[
https://issues.apache.org/jira/browse/HADOOP-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803082#comment-15803082
]
Mingliang Liu edited comment on HADOOP-13931 at 1/6/17 8:58 AM:
----------------------------------------------------------------
[~cnauroth],
I can not reproduce the test failures above. I found they failed last time
because I ran the tests both on my local machine and the AWS EC2 vm using the
same S3 bucket at the same time, which is not supported I believe. Sorry for
the last confusing comment.
I re-ran the tests multiple times (both on EC2 and my local machine) and got
the following report, showing the patch is good as we know the failing ones are
known and unrelated.
{code}
$ mvn -Dtest=none -Dit.test='ITestS3A*' -q -Dscale clean verify
Results :
Tests in error:
ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » IO Failed to
instan...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:669->FileSystemContractBaseTest.rename:525
» AWSServiceIO
ITestS3ACredentialsInURL.testInvalidCredentialsFail:127 » AccessDenied
s3a://m...
Tests run: 321, Failures: 0, Errors: 3, Skipped: 10
{code}
was (Author: liuml07):
[~cnauroth],
I can not reproduce the test failures above. I found they failed last time
because I ran the tests both on my local machine and the AWS EC2 vm using the
same S3 bucket, which is not supported. Sorry for the last confusing comment.
I ran the tests multiple times (both on EC2 and my local machine) and got the
following report, showing the patch is good as we know the failing ones are
known and unrelated.
{code}
$ mvn -Dtest=none -Dit.test='ITestS3A*' -q -Dscale clean verify
Results :
Tests in error:
ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » IO Failed to
instan...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:669->FileSystemContractBaseTest.rename:525
» AWSServiceIO
ITestS3ACredentialsInURL.testInvalidCredentialsFail:127 » AccessDenied
s3a://m...
Tests run: 321, Failures: 0, Errors: 3, Skipped: 10
{code}
> S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()
> -----------------------------------------------------------
>
> Key: HADOOP-13931
> URL: https://issues.apache.org/jira/browse/HADOOP-13931
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Rajesh Balamohan
> Assignee: Mingliang Liu
> Priority: Minor
> Attachments: HADOOP-13931-HADOOP-13345.000.patch,
> HADOOP-13931-HADOOP-13345.001.patch
>
>
> Using {{batchWriteItem}} might be performant in
> {{DynamoDBMetadataStore#put(DirListingMetadata meta)}} and
> {{DynamoDBMetadataStore#put(PathMetadata meta)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
