[
https://issues.apache.org/jira/browse/SPARK-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16291404#comment-16291404
]
Imran Rashid commented on SPARK-14822:
--------------------------------------
[~mgummelt] is this still relevant? seems like dynamic allocation on mesos
already covers this (not precisely the same but pretty close)
> Add lazy executor startup to Mesos Scheduler
> --------------------------------------------
>
> Key: SPARK-14822
> URL: https://issues.apache.org/jira/browse/SPARK-14822
> Project: Spark
> Issue Type: Task
> Components: Mesos
> Reporter: Michael Gummelt
>
> As we deprecate fine-grained mode, we need to make sure we have alternative
> solutions for its benefits.
> Its two benefits are:
> 0. lazy executor startup
> In fine-grained mode, executors are brought up only as tasks are scheduled.
> This means that a user doesn't have to set {{spark.cores.max}} to ensure
> that the app doesn't consume all resources in the cluster.
> 1. relinquishing cores
> As Spark tasks terminate, the mesos task it was bound to terminates as
> well, thus relinquishing the cores assigned to it.
> I'd like to add {{0.}} to coarse-grained mode, possibly enabled with a
> configuration param. If https://issues.apache.org/jira/browse/MESOS-1279
> ever happens, we can add {{1.}} as well.
> cc [~tnachen] [~dragos] [~skonto] [~andrewor14]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
