[
https://issues.apache.org/jira/browse/SPARK-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-11280.
----------------------------------
Resolution: Incomplete
> Mesos cluster deployment using only one node
> --------------------------------------------
>
> Key: SPARK-11280
> URL: https://issues.apache.org/jira/browse/SPARK-11280
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Affects Versions: 1.5.1, 1.6.0
> Reporter: Iulian Dragos
> Priority: Major
> Labels: bulk-closed
> Attachments: Screen Shot 2015-10-23 at 11.37.43.png
>
>
> I submit the SparkPi example in Mesos cluster mode, and I notice that all
> tasks fail except the ones that run on the same node as the driver. The
> others fail with
> {code}
> sh: 1:
> /tmp/mesos/slaves/1521e408-d8fe-416d-898b-3801e73a8293-S0/frameworks/1521e408-d8fe-416d-898b-3801e73a8293-0003/executors/driver-20151023113121-0006/runs/2abefd29-7386-4d81-a025-9d794780db23/spark-1.5.0-bin-hadoop2.6/bin/spark-class:
> not found
> {code}
> The path exists only on the machine that launched the driver, and the sandbox
> of the executor where this task died is completely empty.
> I launch the task like this:
> {code}
> $ spark-submit --deploy-mode cluster --master mesos://sagitarius.local:7077
> --conf
> spark.executor.uri="ftp://sagitarius.local/ftp/spark-1.5.0-bin-hadoop2.6.tgz";
> --conf spark.mesos.coarse=true --class org.apache.spark.examples.SparkPi
> ftp://sagitarius.local/ftp/spark-examples-1.5.0-hadoop2.6.0.jar
> Using Spark's default log4j profile:
> org/apache/spark/log4j-defaults.properties
> 15/10/23 11:31:21 INFO RestSubmissionClient: Submitting a request to launch
> an application in mesos://sagitarius.local:7077.
> 15/10/23 11:31:21 INFO RestSubmissionClient: Submission successfully created
> as driver-20151023113121-0006. Polling submission state...
> 15/10/23 11:31:21 INFO RestSubmissionClient: Submitting a request for the
> status of submission driver-20151023113121-0006 in
> mesos://sagitarius.local:7077.
> 15/10/23 11:31:21 INFO RestSubmissionClient: State of driver
> driver-20151023113121-0006 is now QUEUED.
> 15/10/23 11:31:21 INFO RestSubmissionClient: Server responded with
> CreateSubmissionResponse:
> {
> "action" : "CreateSubmissionResponse",
> "serverSparkVersion" : "1.5.0",
> "submissionId" : "driver-20151023113121-0006",
> "success" : true
> }
> {code}
> I can see the driver in the Dispatcher UI and the job succeeds eventually,
> but running only on the node where the driver was launched (see attachment).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
