[
https://issues.apache.org/jira/browse/HADOOP-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645031#comment-16645031
]
Steve Loughran commented on HADOOP-15837:
-----------------------------------------
Patch 002; fix checkstyle.
Tested S3 ireland with " -Ds3guard -Ddynamodb"; failures in unrelated issues as
discussed and covered elsewhere
* this patch is ready for review
* I plan to backport to 2.10-3.9 the change to the case statement as that
addresses the key issue of capacity change while S3Guard is in use, which
autoscale will implicitly do in high-load situations (i.e. use S3Guard heavily,
DDB triggers scale up, S3Guard fails)
+ [~mackrorysd] : can you look @ this. It's a serious issue which will surface
in the wild
Errors
{code}
[ERROR] Errors:
[ERROR]
ITestS3GuardConcurrentOps.testConcurrentTableCreations:166->deleteTable:77 »
...
[ERROR] ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle:315 »
ResourceInUse Atte...
[ERROR]
ITestS3GuardToolDynamoDB>AbstractS3GuardToolTestBase.testSetCapacityFailFastIfNotGuarded:330->AbstractS3GuardToolTestBase.lambda$testSetCapacityFailFastIfNotGuarded$2:331->AbstractS3GuardToolTestBase.run:115
» FileNotFound
{code}
> DynamoDB table Update can fail S3A FS init
> ------------------------------------------
>
> Key: HADOOP-15837
> URL: https://issues.apache.org/jira/browse/HADOOP-15837
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Environment: s3guard test with small capacity (10) but autoscale
> enabled & multiple consecutive parallel test runs executed...this seems to
> have been enough load to trigger the state change
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-15837-001.patch, HADOOP-15837-002.patch
>
>
> When DDB autoscales a table, it goes into an UPDATING state. The
> waitForTableActive operation in the AWS SDK doesn't seem to wait long enough
> for this to recover. We need to catch & retry
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
