[
https://issues.apache.org/jira/browse/AMBARI-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated AMBARI-11306:
-----------------------------------------
Description:
We copy tarballs to HDFS in several places. The problem is that during Resource
Manager Start in RU it copies
/usr/hdp/current/tez-client/lib/tez.tar.gz to hdfs:///hdp/apps/{{
hdp_stack_version }}/tez/
for the hdp_stack_version value that comes from running "hdp-select status
hadoop-yarn-resourcemanager".
So the destination is correct (uses the "to" version), but the source is
incorrect because the tez-client symlink is still pointing to the older
version, since tez-client only switches to the newer version when the Clients
are upgraded, which occurs later during the orchestration.
The fix is that during RU, the source location of the tarballs must be
/usr/hdp/{{ hdp_version }}/component/tarball.tar.gz
where {{ hdp_version }} is the version going to.
Another problem is that during RU, HiveServer2 Start is not copying
mapreduce.tar.gz tarball to HDFS. In this Jira, we should also clean up the
tarball source and destination variables to come from a file instead of configs
in cluster-env.
was:
We copy tez.tar.gz to HDFS in several places, including Resource Manager Start.
The problem is that during Resource Manager Start in RU it copies
/usr/hdp/current/tez-client/lib/tez.tar.gz to hdfs:///hdp/apps/{{
hdp_stack_version }}/tez/
for the hdp_stack_version value that comes from running "hdp-select status
hadoop-yarn-resourcemanager".
So the destination is correct (uses the "to" version), but the source is
incorrect because the tez-client symlink is still pointing to the older
version, since tez-client only switches to the newer version when the Clients
are upgraded, which occurs later during the orchestration.
Summary: Customers with Ambari 2.0.0 that performed an RU from say HDP 2.2.0.0
to 2.2.4.2 will not have the correct version of tez.tar.gz uploaded to HDFS.
Impact: Applications that depend on this tarball (Pig, Tez, Hive) will be
running with older classes. This does not appear to be breaking in HDP 2.2,
otherwise customers would have reported it. However, it certainly does have to
be fixed in Ambari 2.1.0 because tez.tar.gz has more significant changes and
does encounter a version incompatibility error.
> RU to copy correct version of tarball, including tez.tar.gz and
> mapreduce.tar.gz
> --------------------------------------------------------------------------------
>
> Key: AMBARI-11306
> URL: https://issues.apache.org/jira/browse/AMBARI-11306
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.0.0
> Reporter: Alejandro Fernandez
> Assignee: Alejandro Fernandez
> Fix For: 2.1.0
>
> Attachments: AMBARI-11306.patch
>
>
> We copy tarballs to HDFS in several places. The problem is that during
> Resource Manager Start in RU it copies
> /usr/hdp/current/tez-client/lib/tez.tar.gz to hdfs:///hdp/apps/{{
> hdp_stack_version }}/tez/
> for the hdp_stack_version value that comes from running "hdp-select status
> hadoop-yarn-resourcemanager".
> So the destination is correct (uses the "to" version), but the source is
> incorrect because the tez-client symlink is still pointing to the older
> version, since tez-client only switches to the newer version when the Clients
> are upgraded, which occurs later during the orchestration.
> The fix is that during RU, the source location of the tarballs must be
> /usr/hdp/{{ hdp_version }}/component/tarball.tar.gz
> where {{ hdp_version }} is the version going to.
> Another problem is that during RU, HiveServer2 Start is not copying
> mapreduce.tar.gz tarball to HDFS. In this Jira, we should also clean up the
> tarball source and destination variables to come from a file instead of
> configs in cluster-env.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
