[
https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871450#comment-15871450
]
Mingliang Liu commented on HADOOP-13345:
----------------------------------------
I propose we merge from trunk again. I have fixed the conflicts so if you vote
up, I'll simply push.
{code}
commit a434d50fe4547f32de7b1fafb3c370a7123cda2d
Merge: 8b37b6a96c 02c549484a
Author: Mingliang Liu <[email protected]>
Date: Thu Feb 16 22:38:55 2017 -0800
Merge branch 'trunk' into HADOOP-13345
After HADOOP-14040, we use shaded aws-sdk uber-JAR so don't have to
bring DynamoDB dependency explicitly. However, for tests we do need the
DynamoDBLocal dependency from its Maven repository.
{code}
I got integration tests run against us-west-1. Please confirm as this merge is
major. Thanks,
{code}
Failed tests:
ITestS3AEncryptionSSEC.testCreateFileAndReadWithDifferentEncryptionKey:60
Expected to find 'Forbidden (Service: Amazon S3; Status Code: 403;' but got
unexpected exception:java.nio.file.AccessDeniedException:
s3a://mliu-s3guard/test/testCreateFileAndReadWithDifferentEncryptionKey-0800:
Reopen at position 0 on
s3a://mliu-s3guard/test/testCreateFileAndReadWithDifferentEncryptionKey-0800:
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service:
Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID:
4E0A3A7A0B2D8005), S3 Extended Request ID:
ZKm3w28W57skopifj0wH5p+c8KF1NVzL7ItNG067aK6FNK9dk1kmGrykda/NI4EhtFmN1/bv60c=
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:158)
at
org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:165)
at
org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:291)
at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:374)
Tests in error:
ITestS3ACredentialsInURL.testInstantiateFromURL:86 » InterruptedIO initTable:
...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:669->FileSystemContractBaseTest.rename:525
» AWSServiceIO
{code}
For failing test {{ITestS3AEncryptionSSEC}} I'm not sure it's the caused by the
merge; {{ITestS3ACredentialsInURL}} is known not supported as credentials in
URL are very unsafe.
{{ITestS3AFileSystemContract#testRenameToDirWithSamePrefixAllowed}} I can pass
it 2nd run.
> S3Guard: Improved Consistency for S3A
> -------------------------------------
>
> Key: HADOOP-13345
> URL: https://issues.apache.org/jira/browse/HADOOP-13345
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13345.prototype1.patch, s3c.001.patch,
> S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf,
> S3GuardImprovedConsistencyforS3AV2.pdf
>
>
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a
> stronger consistency model than what is currently offered. The solution
> coordinates with a strongly consistent external store to resolve
> inconsistencies caused by the S3 eventual consistency model.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
