[
https://issues.apache.org/jira/browse/HDFS-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137674#comment-16137674
]
Chris Douglas commented on HDFS-12295:
--------------------------------------
If the subtree under {{/.reserved/bypassExtAttr}} is read-only, that should
address many of the issues that [~daryn] raised. As long as it's only the split
generation that's using this API, that limits the cases that break when this
feature is used.
The requirements for this feature- any user can perform backup-style copies
using distcp- may be too broad. Your objective is to avoid cluttering the
destination namesystem with xattrs from the external attribute provider at the
source. Relying on _all_ users to set this flag correctly is unlikely to
achieve this. What you want is the opposite: copying data between these
clusters, by default, should take the path that reads the raw xattrs.
The less-invasive solutions attempt to relax the requirement that all users run
distcp directly. While the user-facing solution satisfies all the requirements,
it relies on cooperative users. Would it be feasible to add a layer of
indirection in the deployments that need this functionality? If so, then we can
make inter-cluster copies available to all users, without changing the
internals of HDFS.
[Repeating|https://issues.apache.org/jira/browse/HDFS-12202?focusedCommentId=16120861&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16120861]
from HDFS-12202, the {{distcp}} command can be swapped out in 3.x. In
deployments with this requirement, users can contact a service to schedule an
inter-cluster transfer. That backup user could not only be a special-case in
the NameNode plugin, it could also help users avoid copying data from
encryption zones into unprotected clusters (HDFS-6509).
If that's not feasible, can this use case be supported by extending
MAPREDUCE-6007? If the src/dst are under {{/.reserved/raw}}, then omitting the
external attribute provider is reasonable behavior.
> NameNode to support file path prefix /.reserved/bypassExtAttr
> -------------------------------------------------------------
>
> Key: HDFS-12295
> URL: https://issues.apache.org/jira/browse/HDFS-12295
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs, namenode
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Attachments: HDFS-12295.001.patch, HDFS-12295.001.patch
>
>
> Let NameNode to support prefix /.reserved/bypassExtAttr, so client can add
> thisprefix to a path before calling getFileStatus, e.g. /ab/c becomes
> /.reserved/bypassExtAttr/a/b/c. NN will parse the path at the very beginning,
> and bypass external attribute provider if the prefix is there.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
