[
https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2904:
------------------------------
Attachment: hdfs-2904.txt
Attached patch is a first patch at implementing this.
The design is as follows:
- when the client fetches a delegation token, it stores it in the UGI with a
service name corresponding to the "host" portion of the logical URI. So, if the
cluster's logical URI is "hdfs://my-ha-cluster/" then it will store with a DT
service of "my-ha-cluster".
- the TokenRenewer code in DFSClient is extended so that, if the token
corresponds to a configured HA namespace, it will consturct a failover proxy
and renew the token at whichever NN is active
- when constructing the ConfiguredFailoverProxyProvider, it loops over the
addresses for the HA namenodes in the cluster, and for each one, clones the
token to correspond to the correct address
I also did some refactoring/cleanup of the proxy creation code in order to make
this cleaner.
An alternate design I considered was to have the service be a comma-delimited
list of IPC addresses. However, this won't work in the future where we might
not be able to enumerate all possible NNs from the client when we first get the
DT - eg in a zookeeper based setup, the client may only know the address of the
current active.
A question I need to look into:
- when the JT renews delegation tokens on behalf of the job, does it pass the
job's jobconf or the jobtracker's local conf? If the latter, it may not have a
local configuration that has the same clusters configured, which would be
problematic.
I also need to test this on an actual secure cluster with MR to see if it
really works in practice.
> HA: Client support for getting delegation tokens to an HA cluster
> -----------------------------------------------------------------
>
> Key: HDFS-2904
> URL: https://issues.apache.org/jira/browse/HDFS-2904
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, hdfs client, name-node, security
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: hdfs-2904.txt
>
>
> Currently we have server-side support for delegation tokens in HA, and some
> tests to verify it, but the client throws NPEs when trying to fetch a DT.
> This is because the cluster doesn't have a single hostname, but instead a
> logical nameservice name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
