[
https://issues.apache.org/jira/browse/HADOOP-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973669#comment-13973669
]
Jian He commented on HADOOP-10519:
----------------------------------
The following function seems some problem.
In HA mode, getCanonicalServiceName() returns the HA-id (e.g.
ha-hdfs:hostname). Seems both webhdfs token and hdfs token share the same
service name (ha-id), if the webhdfs token is created first, even if we are
issuing a new request for the regular hdfs token, "token =
credentials.getToken(service);" returns non-null (the webHdfsToken), and won't
call getDelegationToken(renewer); to ask for the new regular hdfs token.
{code}
private void collectDelegationTokens(final String renewer,
final Credentials credentials,
final List<Token<?>> tokens)
throws IOException {
final String serviceName = getCanonicalServiceName();
// Collect token of the this filesystem and then of its embedded children
if (serviceName != null) { // fs has token, grab it
final Text service = new Text(serviceName);
LOG.info("serviceName " + serviceName);
Token<?> token = credentials.getToken(service);
if (token == null) {
LOG.info("renewer " + renewer);
token = getDelegationToken(renewer);
if (token != null) {
tokens.add(token);
credentials.addToken(service, token);
}
}
}
// Now collect the tokens from the children
final FileSystem[] children = getChildFileSystems();
if (children != null) {
for (final FileSystem fs : children) {
fs.collectDelegationTokens(renewer, credentials, tokens);
}
}
}
{code}
> In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with
> Client cannot authenticate via:[TOKEN, KERBEROS] error
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10519
> URL: https://issues.apache.org/jira/browse/HADOOP-10519
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Jian He
>
> Opening on [~arpitgupta]'s behalf.
> We observed that, in HDFS HA mode, running Distcp/SLive with webhdfs will
> fail on YARN. In non-HA mode, it'll pass.
> The reason is in HA mode, only webhdfs delegation token is generated for the
> job, but YARN also requires the regular hdfs to do localization,
> log-aggregation etc.
> In non-HA mode, both tokens are generated for the job.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
