[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838317#comment-15838317
]
Steve Loughran commented on HADOOP-13075:
-----------------------------------------
Steve, thanks for taking this up; we'll give you and Frederico joint credit
once this is in.
# Start with trunk only not branch-2. Or work in branch-2 only and plan to
patch forwards...trying to keep 2 branches in sync only adds complications.
This month s3guard branch is the active one, so keep an eye out for any changes
there that would break things —this patch, being simpler, is likelier to get in
first.
# We do have a strict "declare the s3 endpoint you tested against" policy for
all the object stores; that's something you'll be expected to declare on every
patch. Do mention if you saw a transient failure here, always good to keep an
eye on those too. And do run {{ -Dparallel-tests -DtestsThreadCount=8}} for
execution speed.
Overall design seems OK, though it'll need some revisions, In particular, I
don't like all the
f(if(S3AEncryptionMethods.SSE_KMS.getMethod().equals(serverSideEncryptionAlgorithm))
checks everywhere
{{S3AEncryptionMethods}} is an enum; add one for "None" and then you can do
case statements around them; conversion of the config option to an enum value
is a method which can be tucked into the enum itself.
Blocker: The server-side encryption key must be supported; use
{{S3AUtils.getPassword}} for this.
h3. {{S3AFileSystem}}
L 912 : What if the user has selected SSE_C and the server key is blank? That's
downgraded to a no-op, now, yes? I'd rather that was treated
as a failure: if you ask for server-side encryption and don't supply a key, FS
init must fail.
h3. {{S3AEncryptionMethods}}
L 104-119. seems like a spurious IDE-initiated reformat. not needed, as it only
complicates merging branches.
> Add support for SSE-KMS and SSE-C in s3a filesystem
> ---------------------------------------------------
>
> Key: HADOOP-13075
> URL: https://issues.apache.org/jira/browse/HADOOP-13075
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Andrew Olson
> Assignee: Federico Czerwinski
>
> S3 provides 3 types of server-side encryption [1],
> * SSE-S3 (Amazon S3-Managed Keys) [2]
> * SSE-KMS (AWS KMS-Managed Keys) [3]
> * SSE-C (Customer-Provided Keys) [4]
> Of which the S3AFileSystem in hadoop-aws only supports opting into SSE-S3
> (HADOOP-10568) -- the underlying aws-java-sdk makes that very simple [5].
> With native support in aws-java-sdk already available it should be fairly
> straightforward [6],[7] to support the other two types of SSE with some
> additional fs.s3a configuration properties.
> [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html
> [2]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html
> [3] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html
> [4]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
> [5] http://docs.aws.amazon.com/AmazonS3/latest/dev/SSEUsingJavaSDK.html
> [6]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/kms-using-sdks.html#kms-using-sdks-java
> [7] http://docs.aws.amazon.com/AmazonS3/latest/dev/sse-c-using-java-sdk.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
