----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31815/#review75716 -----------------------------------------------------------
Ship it! ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py <https://reviews.apache.org/r/31815/#comment122939> Correct * import. ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py <https://reviews.apache.org/r/31815/#comment122940> Maybe make `tez` and `spark-historyserver` constants since they are reused down below. ambari-server/src/test/python/stacks/2.2/configs/spark-job-history-server.json <https://reviews.apache.org/r/31815/#comment122941> I try to strip out all of the unnecessary JSON when creating test resources so that they are easier to read and require less handling if something about the JSON changes in the future. - Jonathan Hurley On March 6, 2015, 7:29 p.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/31815/ > ----------------------------------------------------------- > > (Updated March 6, 2015, 7:29 p.m.) > > > Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth > Gunturi, and Sid Wagle. > > > Bugs: AMBARI-9954 > https://issues.apache.org/jira/browse/AMBARI-9954 > > > Repository: ambari > > > Description > ------- > > The spark on tez apps fails because tez.tar.gz needs to be copied to HDFS. > Currently, only Pig Service Check and Hive START copy it to HDFS. > > > Diffs > ----- > > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/metainfo.xml > ce8ad7a > > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py > 518352f > ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py > PRE-CREATION > > ambari-server/src/test/python/stacks/2.2/configs/spark-job-history-server.json > PRE-CREATION > > Diff: https://reviews.apache.org/r/31815/diff/ > > > Testing > ------- > > I added a unit test, > ---------------------------------------------------------------------- > Total run:610 > Total errors:0 > Total failures:0 > OK > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ....................................... SUCCESS [6.420s] > [INFO] Apache Ambari Project POM ......................... SUCCESS [0.032s] > [INFO] Ambari Web ........................................ SUCCESS [41.221s] > [INFO] Ambari Views ...................................... SUCCESS [2.345s] > [INFO] Ambari Admin View ................................. SUCCESS [10.019s] > [INFO] Ambari Metrics Common ............................. SUCCESS [1.446s] > [INFO] Ambari Server ..................................... SUCCESS > [33:24.375s] > [INFO] Ambari Agent ...................................... SUCCESS [16.789s] > [INFO] Ambari Client ..................................... SUCCESS [0.038s] > [INFO] Ambari Python Client .............................. SUCCESS [0.257s] > [INFO] Ambari Groovy Client .............................. SUCCESS [9.922s] > [INFO] Ambari Shell ...................................... SUCCESS [0.033s] > [INFO] Ambari Python Shell ............................... SUCCESS [0.036s] > [INFO] Ambari Groovy Shell ............................... SUCCESS [6.632s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 35:01.956s > [INFO] Finished at: Fri Mar 06 16:16:31 PST 2015 > [INFO] Final Memory: 69M/565M > [INFO] > ------------------------------------------------------------------------ > > > I installed Spark and Tez at the same time, and put the Spark History Server > on host 1, and Tez Client on host 2. The installation for Spark History > Server has a dependency on Tez Client, so it also installed it on host 1. The > Spark History Server START command then copied the tarball to > /hdp/apps/${version}/tez/tez.tar.gz , and it will also copy it during a > Rolling Upgrade restart. > > ``` > [root@c6408 ~]# sudo su hdfs -c 'hadoop fs -ls /hdp/apps/2.2.2.0-2538/tez/' > Found 1 items > -r--r--r-- 3 hdfs hadoop 40656789 2015-03-06 02:58 > /hdp/apps/2.2.2.0-2538/tez/tez.tar.gz > > [root@c6408 ~]# su - spark > [spark@c6408 ~]$ /usr/hdp/current/spark-client/bin/spark-submit --class > org.apache.spark.examples.SparkPi --master > execution-context:org.apache.spark.tez.TezJobExecutionContext > /usr/hdp/current/spark-client/lib/spark-examples-1.2.1.2.2.2.0-2538-hadoop2.6.0.2.2.2.0-2538.jar > 3 > > [spark@c6408 ~]$ echo $? > 0 > ``` > > > Thanks, > > Alejandro Fernandez > >
