[
https://issues.apache.org/jira/browse/HADOOP-16118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825970#comment-16825970
]
Gabor Bota commented on HADOOP-16118:
-------------------------------------
Thanks [[email protected]] for backporting this and other fixes.
I reviewed the pr and the patch for 3.2 and it seems ok to me.
While running the tests there is some flakyness in some cases. Result of my
last run:
{noformat}
[ERROR] Failures:
[ERROR]
ITestS3AFileContextMainOperations>FileContextMainOperationsBaseTest.testDeleteRecursively:943
Deleted
[ERROR]
ITestDynamoDBMetadataStore.testProvisionTable:594->Assert.assertEquals:542->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
expected:<200> but was:<100>
[ERROR] Errors:
[ERROR]
ITestS3AContractGetFileStatusV1List>AbstractContractGetFileStatusTest.testComplexDirActions:143->AbstractContractGetFileStatusTest.createTestTree:409
? FileNotFound
[ERROR] ITestS3AFileSystemContract.setUp:67 ? FileNotFound DynamoDB table
'cloudera-de...
[ERROR] ITestS3GuardListConsistency.testConsistentListStatusAfterPut:282 ?
FileNotFound
[ERROR]
ITestS3GuardToolDynamoDB>AbstractS3GuardToolTestBase.testDestroyNoBucket:336->AbstractS3GuardToolTestBase.run:115
? IllegalArgument
[INFO]
[ERROR] Tests run: 684, Failures: 2, Errors: 4, Skipped: 80
{noformat}
{{mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Ds3guard -Ddynamo}}
[This is the console
output|https://gist.github.com/bgaborg/a2b4238c0d08a658eb5aa3341b7bb511]
> S3Guard to support on-demand DDB tables
> ---------------------------------------
>
> Key: HADOOP-16118
> URL: https://issues.apache.org/jira/browse/HADOOP-16118
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16118-branch-3.2-001.patch
>
>
> AWS now supports [on demand DDB
> capacity|https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-no-capacity-planning-and-pay-per-request-pricing/]
>
> This has lowest cost and best scalability, so could be the default capacity.
> + add a new option to set-capacity.
> Will depend on an SDK update: created HADOOP-16117.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
