It seems to be slightly related to this: https://issues.apache.org/jira/browse/SPARK-1340
But in this case, it's not the Task that is failing but the entire executor where the Kafka Receiver resides. 2014-12-16 16:53 GMT+00:00 Luis Ángel Vicente Sánchez < langel.gro...@gmail.com>: > > Dear spark community, > > > We were testing a spark failure scenario where the executor that is > running a Kafka Receiver dies. > > We are running our streaming jobs on top of mesos and we killed the mesos > slave that was running the executor ; a new executor was created on another > mesos-slave but according to the Driver UI, the Kafka receiver location is > still on the dead slave. > > It seems that if the executor where the receiver was running dies, spark > on top of mesos is not able to create a new receiver on a different > executor to continue working. > > Is that a known issue? Is there a way we could ensure that the receiver > would be recreated? > > Kind regards, > > Luis Vicente >
