[
https://issues.apache.org/jira/browse/SPARK-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983314#comment-15983314
]
Michael Gummelt commented on SPARK-20447:
-----------------------------------------
The scheduler doesn't support suppression, no, but it does reject offers for
120s:
https://github.com/apache/spark/blob/master/resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala#L375,
and this is configurable.
With Mesos' 1s offer cycle, this should allow offers to be sent to 119 other
frameworks before being re-offered to Spark.
Is this not sufficient?
> spark mesos scheduler suppress call
> -----------------------------------
>
> Key: SPARK-20447
> URL: https://issues.apache.org/jira/browse/SPARK-20447
> Project: Spark
> Issue Type: Wish
> Components: Mesos
> Affects Versions: 2.1.0
> Reporter: Pavel Plotnikov
> Priority: Minor
>
> The spark mesos scheduler never sends the suppress call to mesos to exclude
> application from Mesos batch allocation process (HierarchicalDRF allocator)
> when spark application is idle and there are no tasks in the queue. As a
> result, the application receives 0% cluster share because of the dynamic
> resource allocation while other applications, that need additional resources,
> can’t receive an offer because they have bigger cluster share that is
> significantly more than 0%
> About suppress call:
> http://mesos.apache.org/documentation/latest/app-framework-development-guide/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
