[
https://issues.apache.org/jira/browse/HDFS-12357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154605#comment-16154605
]
Manoj Govindassamy commented on HDFS-12357:
-------------------------------------------
Thanks for working on this [~yzhangal]. Thanks [~chris.douglas] for the
valuable comments and the alternative proposals. Much appreciated.
My review comments for HDFS-12357.001b.patch.
1. {{FSDirectory.java}}
-- {{getUserFilteredAttributeProvider}} -- line 401 to 407 can be simplified in
a single if block
{noformat}
if (ugi == null || isUserBypassingExtAttrProvider(ugi.getUserName()) {
return null;
}
{noformat}
-- {{initUsersToBypassExtProvider()}} : an user list like "a, b, " can trip the
code to add a null object to the {{usersToBypassExtAttrProvider}}. Probably we
want to verify the trimmed user before adding it to the bypass list.
2. {{hdfs-default.xml}}
-- "..for whom the external attributes provider will be bypassed" - This config
description can have more details like what bypass would mean for the user
operations. Is that only for permission checking or other operations etc.,
3. {{TestINodeAttributeProvider}}
-- Can you please add one non-bypassed user "u4" to the test list of users in
line 239? Basically a true negative case.
-- Check style issues
4. Probably the patch can be renamed to HDFS-12357.006.patch so that any new
reviewers looking at this jira can go straight to the latest patch instead of
suffix versions in the older patch.
> Let NameNode to bypass external attribute provider for special user
> -------------------------------------------------------------------
>
> Key: HDFS-12357
> URL: https://issues.apache.org/jira/browse/HDFS-12357
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Attachments: HDFS-12357.001a.patch, HDFS-12357.001b.patch,
> HDFS-12357.001.patch, HDFS-12357.002.patch, HDFS-12357.003.patch,
> HDFS-12357.004.patch, HDFS-12357.005.patch
>
>
> This is a third proposal to solve the problem described in HDFS-12202.
> The problem is, when we do distcp from one cluster to another (or within the
> same cluster), in addition to copying file data, we copy the metadata from
> source to target. If external attribute provider is enabled, the metadata may
> be read from the provider, thus provider data read from source may be saved
> to target HDFS.
> We want to avoid saving metadata from external provider to HDFS, so we want
> to bypass external provider when doing the distcp (or hadoop fs -cp)
> operation.
> Two alternative approaches were proposed earlier, one in HDFS-12202, the
> other in HDFS-12294. The proposal here is the third one.
> The idea is, we introduce a new config, that specifies a special user (or a
> list of users), and let NN bypass external provider when the current user is
> a special user.
> If we run applications as the special user that need data from external
> attribute provider, then it won't work. So the constraint on this approach
> is, the special users here should not run applications that need data from
> external provider.
> Thanks [~asuresh] for proposing this idea and [~chris.douglas], [~daryn],
> [~manojg] for the discussions in the other jiras.
> I'm creating this one to discuss further.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
