[
https://issues.apache.org/jira/browse/SPARK-11759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011082#comment-15011082
]
Luis Alves edited comment on SPARK-11759 at 11/18/15 2:30 PM:
--------------------------------------------------------------
I just set all the defaults in the spark-defaults.conf like this:
spark.driver.memory 800m
spark.executor.memory 800m
spark.task.cpus 1
spark.mesos.mesosExecutor.cores 1
spark.mesos.coarse false
spark.mesos.executor.docker.image myImage
spark.mesos.executor.home /opt/spark
And then do:
bin/spark-submit --deploy-mode cluster --class myClass
file:/path-to-jar/Test.jar
was (Author: lmtjalves):
I just set all the defaults in the spark-defaults.conf like this:
```
spark.driver.memory 800m
spark.executor.memory 800m
spark.task.cpus 1
spark.mesos.mesosExecutor.cores 1
spark.mesos.coarse false
spark.mesos.executor.docker.image myImage
spark.mesos.executor.home /opt/spark
```
And then do:
bin/spark-submit --deploy-mode cluster --class myClass
file:/path-to-jar/Test.jar
> Spark task on mesos with docker fails with sh: 1: /opt/spark/bin/spark-class:
> not found
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-11759
> URL: https://issues.apache.org/jira/browse/SPARK-11759
> Project: Spark
> Issue Type: Question
> Components: Deploy, Mesos
> Reporter: Luis Alves
>
> I'm using Spark 1.5.1 and Mesos 0.25 in cluster mode. I've the
> spark-dispatcher running, and run spark-submit. The driver is launched, but
> it fails because it seems that the task it launches fails.
> In the logs of the launched task I can see the following error:
> sh: 1: /opt/spark/bin/spark-class: not found
> I checked my docker image and the /opt/spark/bin/spark-class exists. I then
> noticed that it's using sh, therefore I tried to run (in the docker image)
> the following:
> sh /opt/spark/bin/spark-class org.apache.spark.deploy.master.Master
> It fails with the following error:
> spark-class: 73: spark-class: Syntax error: "(" unexpected
> Is this an error in Spark?
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
