[
https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672159#comment-15672159
]
Li Lu commented on MAPREDUCE-6565:
----------------------------------
bq. job.xml does override the tarball configs in almost every way except this
security setting because of the way that setting is loaded
To me this pretty much reveals the nature of a bug. Normally users would expect
to have per-job configs override everything else, but this does not hold with
the use ip setting. So one possible to fix this might be passing in the map
reduce job configuration in security util, instead of using its own?
In SecurityUtil, this limits the default behavior of use ip to be a newly
created Configuration object, which may not be consistent with MR's job
specific setting:
{code}
static {
setConfigurationInternal(new Configuration());
}
{code}
And there is one API for this class to set the configuration used in
SecurityUtil:
{code}
@InterfaceAudience.Public
@InterfaceStability.Evolving
public static void setConfiguration(Configuration conf) {
LOG.info("Updating Configuration");
setConfigurationInternal(conf);
}
{code}
So what we can do is to use the MR app's config to set this configuration?
> Configuration to use host name in delegation token service is not read from
> job.xml during MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6565
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Nauroth
> Assignee: Li Lu
>
> By default, the service field of a delegation token is populated based on
> server IP address. Setting {{hadoop.security.token.service.use_ip}} to
> {{false}} changes this behavior to use host name instead of IP address.
> However, this configuration property is not read from job.xml. Instead, it's
> read from a separate {{Configuration}} instance created during static
> initialization of {{SecurityUtil}}. This does not work correctly with
> MapReduce jobs if the framework is distributed by setting
> {{mapreduce.application.framework.path}} and the
> {{mapreduce.application.classpath}} is isolated to avoid reading
> core-site.xml from the cluster nodes. MapReduce tasks will fail to
> authenticate to HDFS, because they'll try to find a delegation token based on
> the NameNode IP address, even though at job submission time the tokens were
> generated using the host name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
