[
https://issues.apache.org/jira/browse/SPARK-13915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-13915.
-------------------------------
Resolution: Not A Problem
> Allow bin/spark-submit to be called via symbolic link
> -----------------------------------------------------
>
> Key: SPARK-13915
> URL: https://issues.apache.org/jira/browse/SPARK-13915
> Project: Spark
> Issue Type: Improvement
> Components: Spark Submit
> Environment: CentOS 6.6
> Tarbal spark distribution and CDH-5.x.x Spark version (both)
> Reporter: Rafael Pecin Ferreira
> Priority: Minor
>
> We have a CDH-5 cluster that comes with spark-1.5.0 and we needed to use
> spark-1.5.1 for bug fix issues.
> When I set up the spark (out of the CDH box) to the system alternatives, it
> created a sequence of symbolic links to the target spark installation.
> When I tried to run spark-submit, the bash process call the target with "$0"
> as /usr/bin/spark-submit, but this script use the "$0" variable to locate its
> deps and I was facing this messages:
> [hdfs@server01 ~]$ env spark-submit
> ls: cannot access /usr/assembly/target/scala-2.10: No such file or directory
> Failed to find Spark assembly in /usr/assembly/target/scala-2.10.
> You need to build Spark before running this program.
> I fixed the spark-submit script adding this lines:
> if [ -h "$0" ] ; then
> checklink="$0";
> while [ -h "$checklink" ] ; do
> checklink=`readlink $checklink`
> done
> SPARK_HOME="$(cd "`dirname "$checklink"`"/..; pwd)";
> else
> SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)";
> fi
> It would be very nice if this piece of code be put into the spark-submit
> script to allow us to have multiple spark alternatives on the system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
