[
https://issues.apache.org/jira/browse/HADOOP-10150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987655#comment-13987655
]
Yi Liu commented on HADOOP-10150:
---------------------------------
Steve, thank you for the comments.
About blobstores, I remember you brought out this before, it's very good and
can inspire us. A few shortcomings are 1) it's a third part and standalone
service, increase deployment and management. 2) authentication/authorization
issue, for example integration and management. 3) rely on maturity of
blobstores. I'm not saying it's not good, just compared to xattr, the latter
has more merits.
{quote}
generates a per-file key, encrypts it in the public key of users and admins.
{quote}
Agree, having two layer keys is necessary, for example convenient for key
rotation.
{quote}
It'd be good if the mechanism to store/retrieve keys worked with all the
filesystems -even if they didn't have full xattr support. Maybe this could be
done if the design supported a very simple adapter for each FS, which only
handled the read/write of crypto keys
{quote}
Agree, support all the filesystems is one target, and decouple is a basic rule
for programing.
For xattr, it is widely supported on different OS/FS. If some underlying file
system doesn't have full xattr support, we can have fallback in different ways,
but the interfaces can be xattr interfaces, just the implementation, one way is
to use a blobstores. This will guarantee CFS works efficiently and easily
management on top of most filesystems if they support xattr.
> Hadoop cryptographic file system
> --------------------------------
>
> Key: HADOOP-10150
> URL: https://issues.apache.org/jira/browse/HADOOP-10150
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
> Affects Versions: 3.0.0
> Reporter: Yi Liu
> Assignee: Yi Liu
> Labels: rhino
> Fix For: 3.0.0
>
> Attachments: CryptographicFileSystem.patch, HADOOP cryptographic file
> system-V2.docx, HADOOP cryptographic file system.pdf,
> HDFSDataAtRestEncryptionAlternatives.pdf,
> HDFSDataatRestEncryptionAttackVectors.pdf,
> HDFSDataatRestEncryptionProposal.pdf, cfs.patch, extended information based
> on INode feature.patch
>
>
> There is an increasing need for securing data when Hadoop customers use
> various upper layer applications, such as Map-Reduce, Hive, Pig, HBase and so
> on.
> HADOOP CFS (HADOOP Cryptographic File System) is used to secure data, based
> on HADOOP “FilterFileSystem” decorating DFS or other file systems, and
> transparent to upper layer applications. It’s configurable, scalable and fast.
> High level requirements:
> 1. Transparent to and no modification required for upper layer
> applications.
> 2. “Seek”, “PositionedReadable” are supported for input stream of CFS if
> the wrapped file system supports them.
> 3. Very high performance for encryption and decryption, they will not
> become bottleneck.
> 4. Can decorate HDFS and all other file systems in Hadoop, and will not
> modify existing structure of file system, such as namenode and datanode
> structure if the wrapped file system is HDFS.
> 5. Admin can configure encryption policies, such as which directory will
> be encrypted.
> 6. A robust key management framework.
> 7. Support Pread and append operations if the wrapped file system supports
> them.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
