----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42051/#review115077 -----------------------------------------------------------
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py (line 34) <https://reviews.apache.org/r/42051/#comment175933> Assuming upgrade from 2.3 to 2.4: EU: stop on 2.3, changes desired version to 2.4, restarts on 2.4. Downgrade: hdp_stack_version is still 2.4 RU: rolling restart of all components. change desired version to 2.4 at the beginning. Downgrade: hdp_stack_version is 2.3 Instead, use params.version (e.g, 2.4.0.0-1234), which is the version either upgrading/downgrading to. There's a function to format it in order to remove the build number. compare_versions(format_hdp_stack_version(params.version), '2.4.0.0') ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml (line 110) <https://reviews.apache.org/r/42051/#comment175934> Rebase ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml <https://reviews.apache.org/r/42051/#comment175935> No change to this file. ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml (line 106) <https://reviews.apache.org/r/42051/#comment175936> spark-defaults? ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml (line 118) <https://reviews.apache.org/r/42051/#comment175937> spark-defaults? ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml (line 671) <https://reviews.apache.org/r/42051/#comment175938> Wrong file, should use 2.3->2.4 for nonrolling-upgrade-2.4.xml and upgrade-2.4.xml - Alejandro Fernandez On Jan. 18, 2016, 11:27 p.m., Jeff Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42051/ > ----------------------------------------------------------- > > (Updated Jan. 18, 2016, 11:27 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Saisai Shao, and Sumit > Mohanty. > > > Bugs: AMBARI-14561 > https://issues.apache.org/jira/browse/AMBARI-14561 > > > Repository: ambari > > > Description > ------- > > This is for HDP stack, in HDP stack, we'd like to do following changes for > spark thrift server > - Remove advanced properties except queue name > - Use yarn-client mode > - Enable Dynamic Resource Allocation > - Enable Fair scheduling policy > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py > c3ffc7b > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py > 68240bd > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py > 4b38572 > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py > d4c6732 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml > 34423f9 > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml > 6c0e393 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml > d5e4f78 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml > b8bd1ee > > Diff: https://reviews.apache.org/r/42051/diff/ > > > Testing > ------- > > This patch doesn't solve the issue completly. The weird thing is that the > configuration in HDP 2.3's spark-thrift-sparkconf.xml will be appended to > HDP-2.4's spark-thrift-sparkconf.xml. And what I want is remove some > configuration in 2.3. And actually I make Spark 1.6 of HDP 2.4 extend Spark > 1.2 in common-services, not sure why it still use spark-thrift-sparkconf.xml > in HDP 2.3 > > > Thanks, > > Jeff Zhang > >
