[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099923#comment-16099923
]
Stavros Kontopoulos edited comment on SPARK-15142 at 7/25/17 12:10 PM:
-----------------------------------------------------------------------
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted, a DRIVER_ABORTED exception is fired
and its handled properly for the newly submitted drivers?
In your PR (https://github.com/apache/spark/pull/13143) I see you throw the
exception only when you start the dispatcher and then things fail. This code is
used by the dispatcher as well so if you restart the dispatcher and there is no
master it will fail. That means there is no way to detect the new master while
the dispatcher is still alive, it must be restarted when the new master is up,
correct?
Btw your PR https://github.com/apache/spark/pull/13143 is not merged should we
udpate the state of ticket after that?
was (Author: skonto):
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted, a DRIVER_ABORTED exception is fired
and its handled properly for the newly submitted drivers?
In your PR(https://github.com/apache/spark/pull/13143/files) I see you throw
the exception only when you start the dispatcher and then things fail. This
code is used by the dispatcher as well so if you restart the dispatcher and
there is no master it will fail. That means there is no way to detect the new
master while the dispatcher is still alive, it must be restarted when the new
master is up, correct?
> Spark Mesos dispatcher becomes unusable when the Mesos master restarts
> ----------------------------------------------------------------------
>
> Key: SPARK-15142
> URL: https://issues.apache.org/jira/browse/SPARK-15142
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Mesos
> Reporter: Devaraj K
> Priority: Minor
> Attachments:
> spark-devaraj-org.apache.spark.deploy.mesos.MesosClusterDispatcher-1-stobdtserver5.out
>
>
> While Spark Mesos dispatcher running if the Mesos master gets restarted then
> Spark Mesos dispatcher will keep running and queues up all the submitted
> applications and will not launch them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
