[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[
https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092554#comment-15092554
]
Apache Spark commented on SPARK-7831:
-
User 'nraychaudhuri' has created a pull request for this issue:
https://github.com/apache/spark/pull/10701
> Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
> --
>
> Key: SPARK-7831
> URL: https://issues.apache.org/jira/browse/SPARK-7831
> Project: Spark
> Issue Type: Bug
> Components: Mesos
>Affects Versions: 1.4.0
> Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source)
>Reporter: Luc Bourlier
>
> To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be
> running.
> It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher
> registers as a framework in the Mesos cluster.
> After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the
> application is correctly terminated locally, but the framework is still
> listed as {{active}} in the Mesos dashboard.
> I would expect the framework to be de-registered when the dispatcher is
> stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[
https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083142#comment-15083142
]
Nilanjan Raychaudhuri commented on SPARK-7831:
--
I am working on a possible fix for this. I will submit a pull request soon
> Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
> --
>
> Key: SPARK-7831
> URL: https://issues.apache.org/jira/browse/SPARK-7831
> Project: Spark
> Issue Type: Bug
> Components: Mesos
>Affects Versions: 1.4.0
> Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source)
>Reporter: Luc Bourlier
>
> To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be
> running.
> It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher
> registers as a framework in the Mesos cluster.
> After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the
> application is correctly terminated locally, but the framework is still
> listed as {{active}} in the Mesos dashboard.
> I would expect the framework to be de-registered when the dispatcher is
> stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[
https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009070#comment-15009070
]
Zapi Dev commented on SPARK-7831:
-
Well and what's the state of this issue, do you have any plan to implement it?
> Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
> --
>
> Key: SPARK-7831
> URL: https://issues.apache.org/jira/browse/SPARK-7831
> Project: Spark
> Issue Type: Bug
> Components: Mesos
>Affects Versions: 1.4.0
> Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source)
>Reporter: Luc Bourlier
>
> To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be
> running.
> It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher
> registers as a framework in the Mesos cluster.
> After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the
> application is correctly terminated locally, but the framework is still
> listed as {{active}} in the Mesos dashboard.
> I would expect the framework to be de-registered when the dispatcher is
> stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[
https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14559168#comment-14559168
]
Luc Bourlier commented on SPARK-7831:
-
Ok. Yes it makes sense. It is not because the dispatcher died that the the jobs
in progress should be terminated.
I think it would be good to have an option on the shutdown script, to be able
to terminate the Mesos framework.
Also, is there already a way to reconnect a dispatcher to a running framework?
Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
--
Key: SPARK-7831
URL: https://issues.apache.org/jira/browse/SPARK-7831
Project: Spark
Issue Type: Bug
Components: Mesos
Affects Versions: 1.4.0
Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source)
Reporter: Luc Bourlier
To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be
running.
It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher
registers as a framework in the Mesos cluster.
After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the
application is correctly terminated locally, but the framework is still
listed as {{active}} in the Mesos dashboard.
I would expect the framework to be de-registered when the dispatcher is
stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[
https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14556382#comment-14556382
]
Timothy Chen commented on SPARK-7831:
-
Hi Luc, actually that's the default behavior that I put into the dispatcher
framework, since I expect the dispatcher to be a long running framework and
even when it goes away it's expected to be resumed and all the tasks should be
running.
To really shut down the framework a user can ask Mesos to terminate the
framework via the shutdown REST api call.
I think what we should do here is to make a configuration flag to trigger this
behavior or not, and default I think should be what its currently like.
What you think?
Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
--
Key: SPARK-7831
URL: https://issues.apache.org/jira/browse/SPARK-7831
Project: Spark
Issue Type: Bug
Components: Mesos
Affects Versions: 1.4.0
Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source)
Reporter: Luc Bourlier
To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be
running.
It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher
registers as a framework in the Mesos cluster.
After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the
application is correctly terminated locally, but the framework is still
listed as {{active}} in the Mesos dashboard.
I would expect the framework to be de-registered when the dispatcher is
stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
