Ajay Viswanathan created MESOS-1535:
---------------------------------------
Summary: Pyspark on Mesos scheduler error
Key: MESOS-1535
URL: https://issues.apache.org/jira/browse/MESOS-1535
Project: Mesos
Issue Type: Bug
Affects Versions: 0.18.0, 0.18.1
Environment: Running a Mesos on a cluster of Centos 6.5 machines. 180
GB memory.
Reporter: Ajay Viswanathan
This is an error that I get while running fine-grained PySpark on the mesos
cluster. This comes after running some 200-1000 tasks generally.
Pyspark code:
while True:
sc.parallelize(range(10)).map(lambda n : n*2).collect()
Error log:
org.apache.spark.SparkException: EOF reached before Python server acknowledged
at
org.apache.spark.api.python.PythonAccumulatorParam.addInPlace(PythonR
DD.scala:322)
at
org.apache.spark.api.python.PythonAccumulatorParam.addInPlace(PythonR
DD.scala:293)
at org.apache.spark.Accumulable.$plus$plus$eq(Accumulators.scala:70)
at
org.apache.spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala
:275)
at
org.apache.spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala
:273)
at
scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(
TraversableLike.scala:772)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.sca
la:98)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.sca
la:98)
at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala
:226)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
at
scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.s
cala:771)
at org.apache.spark.Accumulators$.add(Accumulators.scala:273)
at
org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGSched
uler.scala:826)
at
org.apache.spark.scheduler.DAGScheduler.processEvent(DAGScheduler.sca
la:601)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$start$1$$anon$2$$ano
nfun$receive$1.applyOrElse(DAGScheduler.scala:190)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
at akka.actor.ActorCell.invoke(ActorCell.scala:456)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(Abst
ractDispatcher.scala:386)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool
.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:19
79)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThre
ad.java:107)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
