[
https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042470#comment-14042470
]
Alejandro Abdelnur commented on HDFS-6134:
------------------------------------------
[~owen.omalley],
bq. "I’m still -1"
I don’t see a previous -1 in any of the related JIRAs.
During the Hadoop Summit, while talking in person, you advocated for the
layered approach because some concerns on the design. Since then, the design
has been changed to address those specific concerns.
bq. Having a layered file system is a much cleaner approach.
The main drawback of the layered approach is that it is not transparent and it
will break and require modifications to a log of existing applications and
projects that assume HDFS file URIs are {{hdfs://}}.
Also, it will break applications and projects that downcast {{FileSystem}} to
{{DistributedFileSystem}}.
bq. Issues:
I think all the issues you are bringing up are being addressed.
Let me try to Recap on the current status of each one of them.
bq. The user needs to be able move, copy, and distribute the directories
without the key.
Yes, this is possible in the current design. FileSystem will had a new
create()/open() signature to support this, if you have access to the file but
not the key, you can use the new signatures to copy files as per the usecase
you are mentioning.
bq. A critical use case for encryption is when hdfs admins should not have
access to the contents of some files.
Correct, the current design addresses this. HDFS admin has access to files but
not the keys.
bq. We shouldn't change the filesystem API to deal with encryption.
We are doing minor changes to enable the usecases you previously indicated. In
the base {{FileSystem}} these operations delegate to the existing methods, thus
no existing filesystem implementation breaks.
BTW, it is not a hack, but a way to enable new usecases that were not a
requirement before. For example, if we ever do transparent compression in HDFS,
you could need these new versions of the create()/open() to be able to copy
files without decompressing/compressing them.
bq. Each file needs to record the key version and original IV as written up in
the CFS design document.
This is happening already.
> Transparent data at rest encryption
> -----------------------------------
>
> Key: HDFS-6134
> URL: https://issues.apache.org/jira/browse/HDFS-6134
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: security
> Affects Versions: 2.3.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: HDFSDataatRestEncryptionProposal_obsolete.pdf,
> HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
>
>
> Because of privacy and security regulations, for many industries, sensitive
> data at rest must be in encrypted form. For example: the healthcare industry
> (HIPAA regulations), the card payment industry (PCI DSS regulations) or the
> US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can
> be used transparently by any application accessing HDFS via Hadoop Filesystem
> Java API, Hadoop libhdfs C library, or WebHDFS REST API.
> The resulting implementation should be able to be used in compliance with
> different regulation requirements.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
