[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842482#comment-15842482
]
Steve Loughran commented on HADOOP-13075:
-----------------------------------------
Don't worry about breaking our tests, those we can adapt if we are confident
the new behaviour is OK. That test was designed to implicitly verify that the
encryption worked by setting an invalid one. Otherwise, we had no evidence
whatsoever that the property was being passed down.
You can just tweak that test to catch the error in initialization. That way we
can verify that invalid options are being rejected.
This does raise the question of how best to test the encryption work
# we should be able to verify that data written with one key cannot be parsed
if a different fs + key is used to read it.
# we should see what happens if you try to read unencrypted data with an FS
with encryption enabled
# maybe: if a bucket is set up to require encryption, then unencrypted data
cannot be written, encrypted can. This implies that the tester will need a
special bucket for this test & declare it in the configs.
Something to try (now?) is for you to enable encryption in auth-keys.xml and
rerun the full test suite with encryption enabled. We've had hints in the past
you may end up with a different length of data returned in getFileStatus() than
you get when you read in the data, and suspect that it's related to encryption
—though probably client-side, rather than server side. running the entire aws
test suite with encryption turned on helps check these things.
> Add support for SSE-KMS and SSE-C in s3a filesystem
> ---------------------------------------------------
>
> Key: HADOOP-13075
> URL: https://issues.apache.org/jira/browse/HADOOP-13075
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Andrew Olson
> Assignee: Federico Czerwinski
>
> S3 provides 3 types of server-side encryption [1],
> * SSE-S3 (Amazon S3-Managed Keys) [2]
> * SSE-KMS (AWS KMS-Managed Keys) [3]
> * SSE-C (Customer-Provided Keys) [4]
> Of which the S3AFileSystem in hadoop-aws only supports opting into SSE-S3
> (HADOOP-10568) -- the underlying aws-java-sdk makes that very simple [5].
> With native support in aws-java-sdk already available it should be fairly
> straightforward [6],[7] to support the other two types of SSE with some
> additional fs.s3a configuration properties.
> [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html
> [2]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html
> [3] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html
> [4]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
> [5] http://docs.aws.amazon.com/AmazonS3/latest/dev/SSEUsingJavaSDK.html
> [6]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/kms-using-sdks.html#kms-using-sdks-java
> [7] http://docs.aws.amazon.com/AmazonS3/latest/dev/sse-c-using-java-sdk.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
