[
https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670553#comment-15670553
]
Jason Lowe commented on MAPREDUCE-6565:
---------------------------------------
bq. In addition, from usability prospective, I think client setting in job.xml
has the highest priority to overwrite whatever conf in tar ball or not - given
a bit overhead to change config setting in tarball. Isn't it?
job.xml does override the tarball configs in almost every way _except_ this
security setting because of the way that setting is loaded (i.e.: via
Configuration instead of JobConf and job.xml is not a default resource).
bq. If so, this actually belongs to a server-side setting, even it take
effective in our current client side code.
In practice that's essentially how it works in our setup. Since this property
is currently not read from job.xml, it ends up using whatever is in the
core-site.xml that's in the MR tarball. The MR tarball is created by the
cluster admins and so the confs are consistent with the server settings for
that cluster. A user could try overriding this property in their job.xml, but
it wouldn't be used because job.xml is ignored for this specific config. (Yes,
a user could provide their own, custom tarball with custom site files to
override this setting, but they need to know what they're doing if they choose
not to use the default MR tarball provided by the cluster admins.)
Given this is a client-side setting that must be in sync with the server-side
setting it'd be nice if this wasn't possible to get out-of-sync. For example,
the server that granted the token could also communicate which setting must be
used by the client later when the token is used, but that's a significant
change with backward compatibility concerns. Of course there's also backwards
compatibility concerns with adding job.xml as a default resource so that this
property could be fetched from there instead of core-site.xml or
core-default.xml.
> Configuration to use host name in delegation token service is not read from
> job.xml during MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6565
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Nauroth
> Assignee: Li Lu
>
> By default, the service field of a delegation token is populated based on
> server IP address. Setting {{hadoop.security.token.service.use_ip}} to
> {{false}} changes this behavior to use host name instead of IP address.
> However, this configuration property is not read from job.xml. Instead, it's
> read from a separate {{Configuration}} instance created during static
> initialization of {{SecurityUtil}}. This does not work correctly with
> MapReduce jobs if the framework is distributed by setting
> {{mapreduce.application.framework.path}} and the
> {{mapreduce.application.classpath}} is isolated to avoid reading
> core-site.xml from the cluster nodes. MapReduce tasks will fail to
> authenticate to HDFS, because they'll try to find a delegation token based on
> the NameNode IP address, even though at job submission time the tokens were
> generated using the host name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
