[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726784#comment-14726784
]
Iulian Dragos commented on SPARK-4940:
--------------------------------------
Would it make sense to allocate resources in a round-robin fashion? Supposing
Spark gets several offers at the same time, it would have enough info to
balance executors on the available resources (or optionally, define an interval
during which it holds on to resources it receives to accumulate a larger set of
slaves).
The algorithm may proceed by allocating a multiple `spark.task.cores` (below
cap, see SPARK-9873 which might help on its own) on each slave in the set of
resources, until it can't allocate anymore.
> Support more evenly distributing cores for Mesos mode
> -----------------------------------------------------
>
> Key: SPARK-4940
> URL: https://issues.apache.org/jira/browse/SPARK-4940
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Attachments: mesos-config-difference-3nodes-vs-2nodes.png
>
>
> Currently in Coarse grain mode the spark scheduler simply takes all the
> resources it can on each node, but can cause uneven distribution based on
> resources available on each slave.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
