-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31815/
-----------------------------------------------------------
Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth
Gunturi, and Sid Wagle.
Bugs: AMBARI-9954
https://issues.apache.org/jira/browse/AMBARI-9954
Repository: ambari
Description
-------
The spark on tez apps fails because tez.tar.gz needs to be copied to HDFS.
Currently, only Pig Service Check and Hive START copy it to HDFS.
Diffs
-----
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/metainfo.xml
ce8ad7a
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
518352f
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py
PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/spark-job-history-server.json
PRE-CREATION
Diff: https://reviews.apache.org/r/31815/diff/
Testing
-------
I added a unit test,
----------------------------------------------------------------------
Total run:610
Total errors:0
Total failures:0
OK
I installed Spark and Tez at the same time, and put the Spark History Server on
host 1, and Tez Client on host 2. The installation for Spark History Server has
a dependency on Tez Client, so it also installed it on host 1. The Spark
History Server START command then copied the tarball to
/hdp/apps/${version}/tez/tez.tar.gz , and it will also copy it during a Rolling
Upgrade restart.
```
[root@c6408 ~]# sudo su hdfs -c 'hadoop fs -ls /hdp/apps/2.2.2.0-2538/tez/'
Found 1 items
-r--r--r-- 3 hdfs hadoop 40656789 2015-03-06 02:58
/hdp/apps/2.2.2.0-2538/tez/tez.tar.gz
[root@c6408 ~]# su - spark
[spark@c6408 ~]$ /usr/hdp/current/spark-client/bin/spark-submit --class
org.apache.spark.examples.SparkPi --master
execution-context:org.apache.spark.tez.TezJobExecutionContext
/usr/hdp/current/spark-client/lib/spark-examples-1.2.1.2.2.2.0-2538-hadoop2.6.0.2.2.2.0-2538.jar
3
[spark@c6408 ~]$ echo $?
0
```
Thanks,
Alejandro Fernandez
