[
https://issues.apache.org/jira/browse/HADOOP-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254445#comment-16254445
]
Xiao Chen commented on HADOOP-14104:
------------------------------------
bq. I don't know why do you need the same fs.defaultFS to make cross cluster
work.
As I said this is arguably a misconfiguration, and for the time being changing
a cluster's nameservice worked. We also suggested that application to manage
the nameservices locally so it can work even with this jira and the same name
service.
In practice, it's possible to have 2 separate existing clusters initially, and
later on have the need to talk to each other. So I think it's worth sharing
here for this potential problem - which happened in reality on production
clusters after an upgrade (albeit a creative application). Good admin
instruction is to plan for this ahead of time, and change one of the name
service in a maintenance window. Otherwise, they would meet this failure.
At the root of the issue, the assumption that the same UGI will be used by
downstream applications to contact namespaces with different name service may
not always be true.
bq. can we use a unique id (e.g., guid)
I thought along similar lines too. But not sure what's the best definition for
the 'key' in the credential map. Maybe an id for the dfsclient together with
the nameservice would solve the problem, but also with the trade off that
different dfsclients accessing really the same nameservice would result in
multiple entries in the UGI's credentials.
It's not clear to me how much performance gain this caching in the UGI gives
us. DFSClients already caches {{serverDefaults}} for 1 hour (The '2.' in
{{getKeyProviderUri}}). Is that sufficient?
> Client should always ask namenode for kms provider path.
> --------------------------------------------------------
>
> Key: HADOOP-14104
> URL: https://issues.apache.org/jira/browse/HADOOP-14104
> Project: Hadoop Common
> Issue Type: Improvement
> Components: kms
> Reporter: Rushabh S Shah
> Assignee: Rushabh S Shah
> Fix For: 2.9.0, 3.0.0-alpha4, 2.8.2
>
> Attachments: HADOOP-14104-branch-2.8.patch,
> HADOOP-14104-branch-2.patch, HADOOP-14104-trunk-v1.patch,
> HADOOP-14104-trunk-v2.patch, HADOOP-14104-trunk-v3.patch,
> HADOOP-14104-trunk-v4.patch, HADOOP-14104-trunk-v5.patch,
> HADOOP-14104-trunk.patch
>
>
> According to current implementation of kms provider in client conf, there can
> only be one kms.
> In multi-cluster environment, if a client is reading encrypted data from
> multiple clusters it will only get kms token for local cluster.
> Not sure whether the target version is correct or not.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
