-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41542/
-----------------------------------------------------------
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-14430
https://issues.apache.org/jira/browse/AMBARI-14430
Repository: ambari
Description
-------
**Problem**:
Wildcards in packages bring and bunch of problems connected with RU and normal
installs.
a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
installed, but hadoop_2_2_0_1234 fails, we can no longer install master
package.
b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
installed before hive_2_2_*, we get problems with inability to install. I've
bumped into a couple of different situations like this. (storm_2_2* and
storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
are real bugs which we had to hack out
c) We install a lot of packages which are not needed by default like hadoop-
2_2_...fuse, just by asking hadoop_2_2_*.
d) Situtations like 47931 can happen in RU.
e) The bug in RU where we add a service after install_packages was triggered.
**Proposal:**
1\. Replace * by ${hdp_version} in metainfo.xml.
E.g.:
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_2_3_*</name>
</package>
<package>
<name>phoenix_2_3_*</name>
</package>
</packages>
</osSpecific>
->
<osSpecific>
<osFamily>redhat7,redhat6,suse11</osFamily>
<packages>
<package>
<name>hbase_${hdp_version}</name>
</package>
<package>
<name>hbase_${hdp_version}-regionserver</name>
</package>
<package>
<name>phoenix_${hdp_version}</name>
</package>
</packages>
</osSpecific>
Note: this step will also let us not override packages in metainfo.xml with
every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.
2\. During INSTALL action (script.py) or during install packages
(install_packages.py) get the version we are currently using from hdp-select.
e.g. for centos:
[root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
2.3.4.0
[root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
3204 # a bit slow for every INSTALL task
Another better aproach is to install hdp-select and than use rpm/dpkg, which
is faster than yum/zypper/apt-get, considering we will do this on every
INSTALL task this is important.
[root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select
| sed -e 's/\.el[0-9]//g' # faster approach
2.3.4.0-3204
u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}'
2.3.4.0-3475
Vs other approaches:
I think it's the only possible approach to get the version here, sending
version which is specified by user during creating of a version. Seems to be a
bad idea as the user can specify wrong version. Also we will need to use this
during fresh install, where we don't have it specified at all.
3\. Replace ${hdp_version} with the version we got in step 2.
I also discussed this with dlysnichenko this seems to be feasible in terms of
RU/EU.
The overall idea was discussed in 47931
cc smohanty
Diffs
-----
ambari-common/src/main/python/resource_management/core/providers/package/apt.py
70a553d
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
87938ad
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
1fbc9bf
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
6dbcef5
ambari-common/src/main/python/resource_management/libraries/functions/packages_analyzer.py
7f34cc1
ambari-common/src/main/python/resource_management/libraries/script/script.py
c954a15
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_service.py
0d9687b
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
d39ccbf
ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/metainfo.xml
fe47276
ambari-server/src/main/resources/stacks/HDP/2.2/services/FLUME/metainfo.xml
4fa4ff7
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/metainfo.xml
71297b1
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml
0366758
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/metainfo.xml
d59b89e
ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/metainfo.xml
aa164e3
ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/metainfo.xml
481fca9
ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/metainfo.xml
db7e025
ambari-server/src/main/resources/stacks/HDP/2.2/services/PIG/metainfo.xml
5472f43
ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/metainfo.xml
0c457f7
ambari-server/src/main/resources/stacks/HDP/2.2/services/SLIDER/metainfo.xml
47bb207
ambari-server/src/main/resources/stacks/HDP/2.2/services/SPARK/metainfo.xml
5cb598e
ambari-server/src/main/resources/stacks/HDP/2.2/services/SQOOP/metainfo.xml
e7ff122
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/metainfo.xml
ccf765b
ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/metainfo.xml
b0ccddd
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml
189defc
ambari-server/src/main/resources/stacks/HDP/2.2/services/ZOOKEEPER/metainfo.xml
d0b37cf
ambari-server/src/main/resources/stacks/HDP/2.3/services/ACCUMULO/metainfo.xml
b19aac1
ambari-server/src/main/resources/stacks/HDP/2.3/services/ATLAS/metainfo.xml
c3a3c81
ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/metainfo.xml
6fdad8b
ambari-server/src/main/resources/stacks/HDP/2.3/services/FLUME/metainfo.xml
5031f85
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metainfo.xml
a37378c
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml
d103caf
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/metainfo.xml
b6b6bb8
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/metainfo.xml
330b542
ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/metainfo.xml
d1172b1
ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml
2fe0ed0
ambari-server/src/main/resources/stacks/HDP/2.3/services/PIG/metainfo.xml
8ba2426
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml
bc52e85
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/metainfo.xml
b1a5584
ambari-server/src/main/resources/stacks/HDP/2.3/services/SLIDER/metainfo.xml
8287b50
ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml
2cf290b
ambari-server/src/main/resources/stacks/HDP/2.3/services/SQOOP/metainfo.xml
1290cde
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/metainfo.xml
00af620
ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/metainfo.xml
7720a3d
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/metainfo.xml
088bb16
ambari-server/src/main/resources/stacks/HDP/2.3/services/ZOOKEEPER/metainfo.xml
c2fe742
ambari-server/src/main/resources/stacks/HDP/2.4/services/ACCUMULO/metainfo.xml
9e82a82
ambari-server/src/main/resources/stacks/HDP/2.4/services/ATLAS/metainfo.xml
158ef8c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FALCON/metainfo.xml
183d55c
ambari-server/src/main/resources/stacks/HDP/2.4/services/FLUME/metainfo.xml
b7cd3d1
ambari-server/src/main/resources/stacks/HDP/2.4/services/HBASE/metainfo.xml
a1c2bd0
ambari-server/src/main/resources/stacks/HDP/2.4/services/HDFS/metainfo.xml
ce3fb95
ambari-server/src/main/resources/stacks/HDP/2.4/services/HIVE/metainfo.xml
5ad52b6
ambari-server/src/main/resources/stacks/HDP/2.4/services/KAFKA/metainfo.xml
90bc472
ambari-server/src/main/resources/stacks/HDP/2.4/services/KNOX/metainfo.xml
e465d42
ambari-server/src/main/resources/stacks/HDP/2.4/services/OOZIE/metainfo.xml
2f09e88
ambari-server/src/main/resources/stacks/HDP/2.4/services/PIG/metainfo.xml
9ae1f42
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml
d76ad5c
ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER_KMS/metainfo.xml
a27bf31
ambari-server/src/main/resources/stacks/HDP/2.4/services/SLIDER/metainfo.xml
ff998f0
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml
f4880e2
ambari-server/src/main/resources/stacks/HDP/2.4/services/SQOOP/metainfo.xml
721cc33
ambari-server/src/main/resources/stacks/HDP/2.4/services/STORM/metainfo.xml
36b01e4
ambari-server/src/main/resources/stacks/HDP/2.4/services/TEZ/metainfo.xml
356225e
ambari-server/src/main/resources/stacks/HDP/2.4/services/YARN/metainfo.xml
c61a270
ambari-server/src/main/resources/stacks/HDP/2.4/services/ZOOKEEPER/metainfo.xml
4bf4198
ambari-server/src/test/python/custom_actions/TestInstallPackages.py 6d8c2f0
ambari-server/src/test/python/custom_actions/configs/install_packages_config.json
50e327a
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 5a95fe2
Diff: https://reviews.apache.org/r/41542/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
