[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099923#comment-16099923
]
Stavros Kontopoulos edited comment on SPARK-15142 at 7/25/17 12:07 PM:
-----------------------------------------------------------------------
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted, a DRIVER_ABORTED exception is fired
and its handled properly for the newly submitted drivers?
In your PR(https://github.com/apache/spark/pull/13143/files) I see you throw
the exception only when you start the dispatcher and then things fail. This
code is used by the dispatcher as well so if you restart the dispatcher and
there is no master it will fail. That means there is no way to detect the new
master while the dispatcher is still alive, it must be restarted when the new
master is up, correct?
was (Author: skonto):
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted,
a DRIVER_ABORTED is fired and its handled properly for the newly submitted
drivers?
In your PR(https://github.com/apache/spark/pull/13143/files) I see you throw
the exception only when you start the dispatcher and then things fail. This is
code used by the dispacther as well so if you restart the dispacther and there
is no master it will fail correct? And that means there is no way to detect the
new master the dispatcher must be restarted when the new master is up.
> Spark Mesos dispatcher becomes unusable when the Mesos master restarts
> ----------------------------------------------------------------------
>
> Key: SPARK-15142
> URL: https://issues.apache.org/jira/browse/SPARK-15142
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Mesos
> Reporter: Devaraj K
> Priority: Minor
> Attachments:
> spark-devaraj-org.apache.spark.deploy.mesos.MesosClusterDispatcher-1-stobdtserver5.out
>
>
> While Spark Mesos dispatcher running if the Mesos master gets restarted then
> Spark Mesos dispatcher will keep running and queues up all the submitted
> applications and will not launch them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
