[
https://issues.apache.org/jira/browse/AMBARI-25551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhiguo Wu updated AMBARI-25551:
-------------------------------
Fix Version/s: 2.8.0
> NodeManager doesn't work after adding a new host to a patch upgraded cluster
> ----------------------------------------------------------------------------
>
> Key: AMBARI-25551
> URL: https://issues.apache.org/jira/browse/AMBARI-25551
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.7.5
> Reporter: Dmytro Vitiuk
> Assignee: Dmitry Lysnichenko
> Priority: Major
> Fix For: 2.8.0, 2.7.6
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> By default "mapreduce.admin.user.env" property for mapreduce2 and
> "tez.am.launch.env"/"tez.task.launch.env" for tez contains a path with a
> specific hdp version, like the next one:
> {code}
> LD_LIBRARY_PATH=/usr/hdp/3.1.5.xx-x/hadoop/lib/native:/usr/hdp/3.1.5.xx-x/hadoop/lib/native/Linux-{{architecture}}-64
> {code}
> As result after patch upgrade (for Tez, MR2, YARN) tez/mapreduce jobs point
> to paths with old versions. And in the case we add new hosts with NodeManager
> only (without any client) we don't have these libraries by the old paths. So
> I am going to change the default value to the next one:
> {code}
> /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-{{architecture}}-64
> {code}
> This path is always exists (because NodeManager installs hadoop-client
> package as dependency).
> Also there is an issue with yarn-shuffle.jar path selection. By default we
> choose the spark client's version in the case a cluster has at least one
> spark client at any host:
> {code}
> /usr/hdp/<sparkClientHDPVersion>/spark2/aux/spark-*-yarn-shuffle.jar
> {code}
> But this way stops working in the case we have hosts without spark client. I
> am going to use yarn's yarn-shuffle.jar path for the same hosts:
> {code}
> /usr/hdp/<yarnClientHDPVersion>/spark2/aux/spark-*-yarn-shuffle.jar
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
