Hi Denny, What do you see in the task log?
Thanks! Tim On Tue, Dec 30, 2014 at 8:25 AM, Denny Lee <[email protected]> wrote: > I've been working with Spark 1.2 and Mesos 0.21.0 and while I have set the > spark.executor.uri within spark-env.sh (and directly within bash as well), > the Mesos slaves do not seem to be able to access the spark tgz file via > HTTP or HDFS as per the message below. > > > 14/12/30 15:57:35 INFO SparkILoop: Created spark context.. > Spark context available as sc. > > scala> 14/12/30 15:57:38 INFO CoarseMesosSchedulerBackend: Mesos task 0 is > now TASK_FAILED > 14/12/30 15:57:38 INFO CoarseMesosSchedulerBackend: Mesos task 1 is now > TASK_FAILED > 14/12/30 15:57:39 INFO CoarseMesosSchedulerBackend: Mesos task 2 is now > TASK_FAILED > 14/12/30 15:57:41 INFO CoarseMesosSchedulerBackend: Mesos task 3 is now > TASK_FAILED > 14/12/30 15:57:41 INFO CoarseMesosSchedulerBackend: Blacklisting Mesos > slave value: "20141228-183059-3045950474-5050-2788-S1" > due to too many failures; is Spark installed on it? > > > I've verified that the Mesos slaves can access both the HTTP and HDFS > locations. I'll start digging into the Mesos logs but was wondering if > anyone had run into this issue before. I was able to get this to run > successfully on Spark 1.1 on GCP - my current environment that I'm > experimenting with is Digital Ocean - perhaps this is in play? > > Thanks! > Denny > >
