[
https://issues.apache.org/jira/browse/HDFS-7568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14333820#comment-14333820
]
Lei (Eddy) Xu commented on HDFS-7568:
-------------------------------------
Hey [~sureshms], thanks a lot for working on this!
I've been doing a bit of research on what it would take to satisfy the
regulatory requirements (SEC Rule 17a-4) for WORM storage, and here's what I
think it would also require:
* It needs a separate audit mechanism. The audit information should be stored
differently from NN's log. It must record all requests to the file under the
immutable directory and has the same retention time as the file. CLIs and APIs
are needed to export this audit log.
* Once a directory is set to WORM mode, neither the system administer or the
user is allowed to turn WORM mode off. Only special personnels have the rights
to delete such directory.
* System time on ANN and SNN can not be mutable and out-of-sync. It requires
that ANN and SNN have a synchronized time service, which can
not be changed by any party. ANN and SNN should also check the time between
them. If one is out of sync, ANN should alarm users and reject further requests.
* It should use {{atime}} to set the retention time and do not allow to modify
{{atime}} later.
* Provide default retention time multiple default retention policies based on
directory patterns. Retention periods shall be specifiable as an offset to the
current system time. Retention periods should also be specifiable as absolute
times.
* Add an "super user" mode to make modifications, which requires special
personnel (e.g., the staffs of the commission and the self-regulatory
organizations) to operate.
* An immutable directory should not be reverted by user, unless the user is the
"super user", as defined in the previous requirement.
* An immutable directory should support creating sub-files and sub-directories.
But sub-directorys must inherit the immutability of its parent directory.
* A file in the immutable directory must satisfy a minimal replica factor (e.g.
>= 2).
Do them make sense to you? Moreover, would you share some information about
your plan on this feature?
Thanks!
> Support immutability (Write-once-read-many) in HDFS
> ---------------------------------------------------
>
> Key: HDFS-7568
> URL: https://issues.apache.org/jira/browse/HDFS-7568
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Affects Versions: 2.7.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
>
> Many regulatory compliance requires storage to support WORM functionality to
> protect sensitive data from being modified or deleted. This jira proposes
> adding that feature to HDFS.
> See the following comment for more description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
