[
https://issues.apache.org/jira/browse/HDFS-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755112#comment-13755112
]
Steve Loughran commented on HDFS-5143:
--------------------------------------
I'm confused about one issue: is there going to be a difference between the
listable length of a file ({{FileSystem.listStatus()}}, and the user-code
visible length of a file:
bq. For example, if the decorated file system is HDFS, the file length stored
in namenode should be the length of the encrypted file, but when getting
FileStatus using CFS API in Map-Reduce, the file length should be the length of
decrypted file.
Or is it that the cfs:// view is consistent across all file stat operations,
seek() etc.?
Either way, I'm curious about how this interacts with quotas. Presumably the
HDFS quota for a specific storage tier applies. This could lead to some
failures converting/copying a file from an unencrypted directory to an
encrypted one.
Finally, are all operations that are atomic today -e.g renaming one directory
under another- going to remain atomic?
> Hadoop cryptographic file system
> --------------------------------
>
> Key: HDFS-5143
> URL: https://issues.apache.org/jira/browse/HDFS-5143
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: security
> Affects Versions: 3.0.0
> Reporter: Yi Liu
> Labels: rhino
> Fix For: 3.0.0
>
> Attachments: HADOOP cryptographic file system.pdf
>
>
> There is an increasing need for securing data when Hadoop customers use
> various upper layer applications, such as Map-Reduce, Hive, Pig, HBase and so
> on.
> HADOOP CFS (HADOOP Cryptographic File System) is used to secure data, based
> on HADOOP “FilterFileSystem” decorating DFS or other file systems, and
> transparent to upper layer applications. It’s configurable, scalable and fast.
> High level requirements:
> 1. Transparent to and no modification required for upper layer
> applications.
> 2. “Seek”, “PositionedReadable” are supported for input stream of CFS if
> the wrapped file system supports them.
> 3. Very high performance for encryption and decryption, they will not
> become bottleneck.
> 4. Can decorate HDFS and all other file systems in Hadoop, and will not
> modify existing structure of file system, such as namenode and datanode
> structure if the wrapped file system is HDFS.
> 5. Admin can configure encryption policies, such as which directory will
> be encrypted.
> 6. A robust key management framework.
> 7. Support Pread and append operations if the wrapped file system supports
> them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
