[
https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677579#comment-15677579
]
Junping Du commented on MAPREDUCE-6565:
---------------------------------------
bq. I think the tarball is muddying the waters here. *-site.xml files should be
treated the same whether they are in a tarball or not. It'd be tricky and messy
to do otherwise. Essentially what we're asking is whether clients should be
able to override any non-final setting in the *-site.xml files with their
job.xml setting, even if that setting is a "server side" property
Agree. Whether clients are allowed to override any non-final setting is
something I originally asking for. But later, I realized that if we disallow
client to override "server-setting" - like case here, tar-ball configuration
will become the final setting for "server-side" configuration. It also means MR
tarball will become various according to cluster settings but not according to
releases. Standing at support-ability prospective, we don't want our users to
touch tarball settings because it is hard to control/monitor these settings by
Ambari or other tools. Otherwise, we have to dig into tarball settings when
weird things happens. In this sense, I think it could be simpler if we don't
differentiate client-side and server-side setting in configuration loading.
bq. The risk of this change is when the client's *-site.xml files do not match
what should be there. For the "server side" settings this has been working
because we've been ignoring job.xml for those. Once we start using job.xml for
even those properties, jobs that were working in the past because we ignored
bad values will break. I don't know offhand how many other properties besides
this one could suddenly change because we start using the client's version of
the property in job.xml when we didn't before.
I think even before, our configuration loading doesn't guarantee we only
loading client-side configuration rather than server-side configuration. Even
for mapred-site.xml, there are history server related settings that belongs to
server-side but you can override a different value in client side which just
doesn't work though. So we still need to rely on code logic to bypass what
settings belongs to server-setting in client side. Also, better documentation
could be helpful for user to differentiate what's client-side and what's
server-side. Our current tricky loading mechanism won't help on both side.
> Configuration to use host name in delegation token service is not read from
> job.xml during MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6565
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Nauroth
> Assignee: Li Lu
>
> By default, the service field of a delegation token is populated based on
> server IP address. Setting {{hadoop.security.token.service.use_ip}} to
> {{false}} changes this behavior to use host name instead of IP address.
> However, this configuration property is not read from job.xml. Instead, it's
> read from a separate {{Configuration}} instance created during static
> initialization of {{SecurityUtil}}. This does not work correctly with
> MapReduce jobs if the framework is distributed by setting
> {{mapreduce.application.framework.path}} and the
> {{mapreduce.application.classpath}} is isolated to avoid reading
> core-site.xml from the cluster nodes. MapReduce tasks will fail to
> authenticate to HDFS, because they'll try to find a delegation token based on
> the NameNode IP address, even though at job submission time the tokens were
> generated using the host name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
