[
https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675073#comment-15675073
]
Jason Lowe commented on MAPREDUCE-6565:
---------------------------------------
bq. Can we make a consensus that everything in tarball's config is not final
(unless we explicitly mark it as final)?
I think the tarball is muddying the waters here. *-site.xml files should be
treated the same whether they are in a tarball or not. It'd be tricky and
messy to do otherwise. Essentially what we're asking is whether clients should
be able to override _any_ non-final setting in the *-site.xml files with their
job.xml setting, even if that setting is a "server side" property.
bq. may be we can simply go ahead to make job.xml as the highest priority
without differentiating client/server settings. Any risk I am missing here?
Making job.xml a default resource accomplishes that proposal if we want to go
that route. Note: Usually job.xml contains the contents of the client-side
*-site.xml files, so as long as those match the cluster we should be good there.
The risk of this change is when the client's *-site.xml files do _not_ match
what should be there. For the "server side" settings this has been working
because we've been ignoring job.xml for those. Once we start using job.xml for
even those properties, jobs that were working in the past because we ignored
bad values will break. I don't know offhand how many other properties besides
this one could suddenly change because we start using the client's version of
the property in job.xml when we didn't before.
I agree with [~gtCarrera9] that it's more consistent and less surprising to
users if job.xml settings override any other settings in the job. However
there are going to be some cases that break when we "fix" it. That's why I'm a
bit hesitant, especially if this is going into 2.x.
> Configuration to use host name in delegation token service is not read from
> job.xml during MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6565
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Nauroth
> Assignee: Li Lu
>
> By default, the service field of a delegation token is populated based on
> server IP address. Setting {{hadoop.security.token.service.use_ip}} to
> {{false}} changes this behavior to use host name instead of IP address.
> However, this configuration property is not read from job.xml. Instead, it's
> read from a separate {{Configuration}} instance created during static
> initialization of {{SecurityUtil}}. This does not work correctly with
> MapReduce jobs if the framework is distributed by setting
> {{mapreduce.application.framework.path}} and the
> {{mapreduce.application.classpath}} is isolated to avoid reading
> core-site.xml from the cluster nodes. MapReduce tasks will fail to
> authenticate to HDFS, because they'll try to find a delegation token based on
> the NameNode IP address, even though at job submission time the tokens were
> generated using the host name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
