[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099923#comment-16099923
]
Stavros Kontopoulos edited comment on SPARK-15142 at 7/25/17 12:05 PM:
-----------------------------------------------------------------------
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted,
a DRIVER_ABORTED is fired and its handled properly for the newly submitted
drivers?
In your PR(https://github.com/apache/spark/pull/13143/files) I see you throw
the exception only when you start the dispatcher and then things fail. This is
code used by the dispacther as well so if you restart the dispacther and there
is no master it will fail correct? And that means there is no way to detect the
new master the dispatcher must be restarted when the new master is up.
was (Author: skonto):
[~devaraj.k] Great, so one question to clarify things, if the dispatcher is
running and then the master is restarted,
a DRIVER_ABORTED is fired and its handled properly for the newly submitted
drivers? In that PR you mention it says: "I think we need to throw the
exception and shutdown the dispatcher." and so later some other PRs like
(https://github.com/apache/spark/pull/13072) handle the shutdown process.
In your PR(https://github.com/apache/spark/pull/13143/files) I see you throw
the exception only when you start the dispatcher and then things fail. This is
code used by the dispacther as well so if you restart the dispacther and there
is no master it will fail correct? And that means there is no way to detect the
new master the dispatcher must be restarted when the new master is up.
> Spark Mesos dispatcher becomes unusable when the Mesos master restarts
> ----------------------------------------------------------------------
>
> Key: SPARK-15142
> URL: https://issues.apache.org/jira/browse/SPARK-15142
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Mesos
> Reporter: Devaraj K
> Priority: Minor
> Attachments:
> spark-devaraj-org.apache.spark.deploy.mesos.MesosClusterDispatcher-1-stobdtserver5.out
>
>
> While Spark Mesos dispatcher running if the Mesos master gets restarted then
> Spark Mesos dispatcher will keep running and queues up all the submitted
> applications and will not launch them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
