----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30733/#review71464 -----------------------------------------------------------
Ship it! I'm ok with this going in now, and fixing the named imports laters. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/configuration/spark-defaults.xml <https://reviews.apache.org/r/30733/#comment117197> May as well remove this property. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py <https://reviews.apache.org/r/30733/#comment117199> Preferable to use named imports instead of * ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py <https://reviews.apache.org/r/30733/#comment117200> From RU perspective, this looks good. We will need hdp-select to support the "spark" component. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py <https://reviews.apache.org/r/30733/#comment117203> Remove all of the unnecessary "pass" calls ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py <https://reviews.apache.org/r/30733/#comment117206> Will need to consider this when switching between versions. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py <https://reviews.apache.org/r/30733/#comment117204> Use named imports. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py <https://reviews.apache.org/r/30733/#comment117208> This may need to be generalized depending on the stack type, e.g., HDP. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py <https://reviews.apache.org/r/30733/#comment117209> Should this use hadoop-client instead? - Alejandro Fernandez On Feb. 6, 2015, 4:38 p.m., Gautam Borad wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30733/ > ----------------------------------------------------------- > > (Updated Feb. 6, 2015, 4:38 p.m.) > > > Review request for Ambari, Jaimin Jetly and Yusaku Sako. > > > Bugs: AMBARI-9494 > https://issues.apache.org/jira/browse/AMBARI-9494 > > > Repository: ambari > > > Description > ------- > > Add support for management for Spark using Ambari. > Support for following components are to be addressed : > 1] Spark Client > 2] Spark Job History Server > > > Diffs > ----- > > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/configuration/spark-defaults.xml > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/configuration/spark-env.xml > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/configuration/spark-log4j-properties.xml > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/configuration/spark-metrics-properties.xml > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/metainfo.xml > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.2/services/SPARK/metainfo.xml > PRE-CREATION > > Diff: https://reviews.apache.org/r/30733/diff/ > > > Testing > ------- > > Did the following testing: > > 1] Tested on RHEL6 single node cluster > 2] Tested on RHEL6 multi-node cluster with Job history server on one node and > spark client on other nodes > 3] Submitted a job using $PARK_HOME/bin/spark-submit and was able to see the > job on Spark History Server > > Please note the TODO in params.py, hostLevelParams/stack_version is returning > 2.2.0.0, even if the HDP version is 2.2.1.0-xxxx > > > Thanks, > > Gautam Borad > >
