[
https://issues.apache.org/jira/browse/MAPREDUCE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-4877:
----------------------------------------
Target Version/s: 2.6.0 (was: 3.0.0, 2.5.0)
> AM doesn't properly support multiple NNs
> ----------------------------------------
>
> Key: MAPREDUCE-4877
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4877
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, job submission
> Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> Yarn/MR clusters assume there's a 1-to-1 correspondence between itself and a
> NN. Certain internal paths like the staging dir, job history,
> intermediate/intermediate-done dirs are resolved relative to the defaultFS.
> The JT used the host's conf which ensured the correct/expected NN. However
> the AM uses the user's job conf, which means the user's defined defaultFS can
> cause the job to use incorrect paths.
> Typically the output path's NN is also the yarn cluster's NN. However
> problems occur when a yarn cluster is servicing multiple NN's (ex. federated
> clusters). The JHS is assuming the AM will write to NN1, whereas the user's
> job conf may be using a defaultFS of NN2 or NN3 which influences where the AM
> writes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
