[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314171#comment-16314171
]
Aaron Fabbri commented on HADOOP-15006:
---------------------------------------
Thanks again for writing this up [~moist]--it is very helpful. I'm in general
agreement with the discussion here.
The length / seek issue is interesting.
Do have any good links for further reading on the crypto algorithms,
particularly the NoPadding variant you mention? (How do lengths and byte
offsets map from the user data to the encrypted stream?)
What are the actual atomicity requirements? Specifically, how do we handle
multiple clients racing to create the same path?
Option 5 (store encryption metadata in Dynamo, but in its own separate table)
sounds good to me. As we discussed offline, data in S3Guard has a different
lifetime (it is not required to be retained, and that policy offers multiple
benefits for S3Guard but would cause data loss for CSE). Also since the scope
of the encryption zone is the bucket, we could get by with a very low
provisioned I/O budget on the Dynamo table and save money, no?
I'm available any time to give a walkthrough of S3Guard's DynamoDB logic or
answer any questions about it.
Also thanks [~xiaochen] and Steve for taking time to look over this.
> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
> ---------------------------------------------------------------
>
> Key: HADOOP-15006
> URL: https://issues.apache.org/jira/browse/HADOOP-15006
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3, kms
> Reporter: Steve Moist
> Priority: Minor
> Attachments: S3-CSE Proposal.pdf
>
>
> This is for the proposal to introduce Client Side Encryption to S3 in such a
> way that it can leverage HDFS transparent encryption, use the Hadoop KMS to
> manage keys, use the `hdfs crypto` command line tools to manage encryption
> zones in the cloud, and enable distcp to copy from HDFS to S3 (and
> vice-versa) with data still encrypted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
