[
https://issues.apache.org/jira/browse/HDFS-12357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongjun Zhang updated HDFS-12357:
---------------------------------
Attachment: HDFS-12357.001a.patch
Hi [~chris.douglas],
Thanks a lot for the good discussion yesterday!
We favor wrapper solution for the better abstraction design. But due to the
extra checking I described in my last comment, I did a revised version 001a on
top of v001 for reference. Mainly introduced a new method and calls to it at
needed places:
{code}
private INodeAttributeProvider getUserFilteredAttributeProvider(
UserGroupInformation ugi) {
if (ugi == null) {
return attributeProvider;
}
if (attributeProvider == null ||
isUserBypassingExtAttrProvider(ugi.getUserName())) {
return null;
}
return attributeProvider;
}
{code}
Could you please take a look at both 005 and 001a? Either one would work for
us, and we can keep improving the solution.
Thanks a lot.
> Let NameNode to bypass external attribute provider for special user
> -------------------------------------------------------------------
>
> Key: HDFS-12357
> URL: https://issues.apache.org/jira/browse/HDFS-12357
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Attachments: HDFS-12357.001a.patch, HDFS-12357.001.patch,
> HDFS-12357.002.patch, HDFS-12357.003.patch, HDFS-12357.004.patch,
> HDFS-12357.005.patch
>
>
> This is a third proposal to solve the problem described in HDFS-12202.
> The problem is, when we do distcp from one cluster to another (or within the
> same cluster), in addition to copying file data, we copy the metadata from
> source to target. If external attribute provider is enabled, the metadata may
> be read from the provider, thus provider data read from source may be saved
> to target HDFS.
> We want to avoid saving metadata from external provider to HDFS, so we want
> to bypass external provider when doing the distcp (or hadoop fs -cp)
> operation.
> Two alternative approaches were proposed earlier, one in HDFS-12202, the
> other in HDFS-12294. The proposal here is the third one.
> The idea is, we introduce a new config, that specifies a special user (or a
> list of users), and let NN bypass external provider when the current user is
> a special user.
> If we run applications as the special user that need data from external
> attribute provider, then it won't work. So the constraint on this approach
> is, the special users here should not run applications that need data from
> external provider.
> Thanks [~asuresh] for proposing this idea and [~chris.douglas], [~daryn],
> [~manojg] for the discussions in the other jiras.
> I'm creating this one to discuss further.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
