[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341776#comment-16341776
]
Steve Moist commented on HADOOP-15006:
--------------------------------------
Ok fixed it. The workflow with block output was causing it to write to disk
encrypted and then when it sent it to S3 it encrypted it again causing it to
decrypt. So there's a small issue with that in some cases. However, now
encryption should work fine for most things. It uses a fixed IV and key to do
the encryption, so any files written to S3 will be automatically
encrypted/decrypted, so we get some free coverage from the unit tests. It's a
quick and dirty prototype so many of the unit tests fail as its not covering
all scenarios. I'm able to upload/download files to S3 using the command line
without issue. When I view the object in S3 gui, it shows up encrypted, but
will automatically decrypt when i do a hdfs get from the cli. Play around with
it and let me know what you think. The CryptoStreams work fine, but the
integration to fully flesh this out into a feature is what we need to really
look at.
> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
> ---------------------------------------------------------------
>
> Key: HADOOP-15006
> URL: https://issues.apache.org/jira/browse/HADOOP-15006
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3, kms
> Reporter: Steve Moist
> Priority: Minor
> Attachments: S3-CSE Proposal.pdf, s3-cse-poc.patch
>
>
> This is for the proposal to introduce Client Side Encryption to S3 in such a
> way that it can leverage HDFS transparent encryption, use the Hadoop KMS to
> manage keys, use the `hdfs crypto` command line tools to manage encryption
> zones in the cloud, and enable distcp to copy from HDFS to S3 (and
> vice-versa) with data still encrypted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
