[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258570#comment-14258570
]
Timothy Chen commented on SPARK-4940:
-------------------------------------
Potentially I think there are two ways to help distribute the allocation:
1) Use static reservation in mesos side, to reserve resources just for Spark
which gurantees the coarse grained mode will use them. The downside is that
when Spark doesn't need these resources it can't be shared with other
frameworks.
2) In the Spark scheduler side we can perhaps have a minimum and maximum cpu
allocation count, so that besides just requiring 1 cpu we also ask to have the
required cores to be within a range, so it's much more evenly allocated.
> Document or Support more evenly distributing cores for Mesos mode
> -----------------------------------------------------------------
>
> Key: SPARK-4940
> URL: https://issues.apache.org/jira/browse/SPARK-4940
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
>
> Currently in Coarse grain mode the spark scheduler simply takes all the
> resources it can on each node, but can cause uneven distribution based on
> resources available on each slave.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
