-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55515/
-----------------------------------------------------------
(Updated Jan. 19, 2017, 8:25 p.m.)
Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and
Vitalyi Brodetskyi.
Changes
-------
Final patch
Addressed comments
Bugs: AMBARI-18739
https://issues.apache.org/jira/browse/AMBARI-18739
Repository: ambari
Description
-------
Also includes code for AMBARI-18738. Perf: Allow components to report their
version using distro-select and change configs using conf-select
In order to run Rolling/Express Upgrade, will need to create the
Rolling/NonRolling Upgrade Packs and potentially config packs if we decide to
do it across major versions.
This means each service will need to support start commands with the
upgrade_type option, have symlinks already in distro-select, etc.
At first, this can be a handful of services.
The goal is to simulate Rolling/Express Upgrade on the PERF stack.
Diffs (updated)
-----
ambari-agent/src/main/python/ambari_agent/PythonExecutor.py d75fc0cef2
ambari-common/src/main/python/resource_management/libraries/script/dummy.py
3dcece501d
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py
d70e87fb1e
ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-INSTALL/scripts/distro-select.py
PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-INSTALL/scripts/hook.py
9eca9ec724
ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-INSTALL/scripts/hook.py
9eca9ec724
ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-INSTALL/scripts/hook.py
9eca9ec724
ambari-server/src/main/resources/stacks/PERF/1.0/metainfo.xml 45a63e5550
ambari-server/src/main/resources/stacks/PERF/1.0/metainfo.xml 45a63e5550
ambari-server/src/main/resources/stacks/PERF/1.0/metainfo.xml 45a63e5550
ambari-server/src/main/resources/stacks/PERF/1.0/repos/repoinfo.xml
c5137fb133
ambari-server/src/main/resources/stacks/PERF/1.0/repos/repoinfo.xml
c5137fb133
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/kerberos.json
3bf5fbd589
ambari-server/src/main/resources/stacks/PERF/1.0/services/GRUMPY/metainfo.xml
15c5cfddd2
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/kerberos.json
ae84eaff8e
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml
53cc11b7eb
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json
24141e8389
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-env.xml
d3dda94fac
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-log4j.xml
880e8837cf
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-logsearch-conf.xml
7c481f29cf
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-policy.xml
8500ee02de
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-site.xml
058facdfe9
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-audit.xml
e518ea0b10
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/ranger-hbase-security.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/kerberos.json
9ed40ef7ed
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml
f1f4b7d1ac
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metrics.json
bf1fa6e7d9
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_client.py
c4943fed3c
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_master.py
3761fcf03a
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/hbase_regionserver.py
101c36fe56
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/phoenix_queryserver.py
76a49d9105
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/scripts/service_check.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/quicklinks/quicklinks.json
5568122034
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/themes/theme.json
7a5732bdac
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/widgets.json
ae47833059
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json
34cea4c2ed
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/core-site.xml
8375be17a7
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-env.xml
3be63be416
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-metrics2.properties.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hadoop-policy.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-log4j.xml
a4eacc247a
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-logsearch-conf.xml
6540c865a4
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-site.xml
995632f339
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-audit.xml
3dc46b3aea
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
deede1c1ba
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ranger-hdfs-security.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ssl-client.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/ssl-server.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/kerberos.json
974a69cbdc
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml
42edc8d493
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metrics.json
c66387d3a1
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/datanode.py
6fc338b1da
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/hdfs_client.py
4280c6cc98
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/journalnode.py
96be63060a
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/namenode.py
c3488e8bcc
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/nfsgateway.py
b750522220
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/service_check.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/snamenode.py
91ce7da9de
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/scripts/zkfc_slave.py
b4310725a3
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/quicklinks/quicklinks.json
a4216e35ac
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/themes/theme.json
6f2b79741b
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/widgets.json
4a645b01dc
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/kerberos.json
45e33f5ec7
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml
5b4dbd4a2d
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/kerberos.json
47a979a8c3
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml
21e685ad80
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/YARN_metrics.json
a66bb34464
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/YARN_widgets.json
4b76a177b6
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json
67cf881caa
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration-mapred/mapred-env.xml
fbd056d2a2
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration-mapred/mapred-site.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/capacity-scheduler.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/ranger-yarn-audit.xml
0352424747
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/ranger-yarn-plugin-properties.xml
97867cc6f8
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/ranger-yarn-policymgr-ssl.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/ranger-yarn-security.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-env.xml
f81080eadf
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-log4j.xml
5e9ac98a18
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-site.xml
2d56e38bd2
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/kerberos.json
f33c07d042
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml
bb2191b267
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/application_timeline_server.py
d179b1f2ca
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/historyserver.py
0570987aa7
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/mapred_service_check.py
66571f8ca6
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/mapred_service_check.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/mapreduce2_client.py
9b1274d685
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/nodemanager.py
21db94fcbe
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/resourcemanager.py
5f6c535186
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/service_check.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/scripts/yarn_client.py
98a7c52b6f
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/quicklinks-mapred/quicklinks.json
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/quicklinks/quicklinks.json
0aca8e3b13
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/themes-mapred/theme.json
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/themes/theme.json
758cf0c969
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json
4af7ae9447
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/kerberos.json
0a64ea526e
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml
57667bb888
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/scripts/service_check.py
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/scripts/zookeeper_client.py
25b2409dcd
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/scripts/zookeeper_server.py
6ab88bb3c1
ambari-server/src/main/resources/stacks/PERF/1.0/upgrades/nonrolling-upgrade-2.0.xml
PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/upgrades/upgrade-2.0.xml
PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/PythonExecutor.sed PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/install_packages.sed
PRE-CREATION
contrib/utils/perf/deploy-gce-perf-cluster.py 8ba7e42e0b
Diff: https://reviews.apache.org/r/55515/diff/
Testing
-------
Tested on my local 3-node cluster (both upgrade and downgrade).
Script for creating Perf cluster downloads Ambari code from Apache git, so did
not manage to test on gce, but script should work too (added required code).
What works:
- deploy of Perf 1.0 cluster
- RU to 2.0 (upgrade and downgrade)
- EU to 2.0 (upgrade and downgrade)
Issues:
- Clean deploy of perf 2.0 stack (dummy component install scripts are hardcoded
with 1.1.1.1 version)
Thanks,
Dmitro Lisnichenko
