[
https://issues.apache.org/jira/browse/MAPREDUCE-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586660#comment-13586660
]
Jerry Chen commented on MAPREDUCE-5025:
---------------------------------------
[~owen.omalley]
The reason we proposed file formats extended with encryption support instead of
something at a lower layer is so the user can selectively apply encryption only
where they feel necessary, only for those MR jobs that require it. This also
helps keep the resulting files and the usage of encryption filesystem agnostic.
Adding transparent encryption to a filesystem is an interesting idea and
something that we also prototyped as part of this work. Perhaps a Common JIRA
for an encrypting filesystem derived from FileSystem would be appropriate? Or
an HDFS JIRA for plugging in compression and crypto codecs to block storage
and transfer? We could look at something like those for follow on work.
> Key Distribution and Management for supporting crypto codec in Map Reduce
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-5025
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5025
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: security
> Affects Versions: trunk
> Reporter: Jerry Chen
> Assignee: Jerry Chen
> Attachments: MAPREDUCE-5025.patch
>
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> This task defines the work to enable Map Reduce to utilize the Crypto Codec
> framework to support encryption and decryption of data during MapReduce Job.
> According to the some real use case and discussions from the community, for
> encryption and decryption files in Map Reduce, we have the following
> requirements:
> 1. Different stages (input, output, intermediate output) should have the
> flexibility to choose whether encrypt or not, as well as which crypto codec
> to use.
> 2. Different stages may have different scheme of providing the keys.
> 3. Different Files (for example, different input files) may have or use
> different keys.
> 4. Support a flexible way of retrieving keys for encryption or decryption.
> So this task defines and provides the framework for supporting these
> requirements as well as the implementations for common use and key retrieving
> scenarios.
> The design document of this part is included in the Hadoop Crypto Design
> attached in HADOOP-9331.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
