[
https://issues.apache.org/jira/browse/SPARK-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411313#comment-15411313
]
Sun Rui commented on SPARK-16944:
---------------------------------
Not quite sure. I think the Mesos scheduler backend can get the preferred
locations of pending task and wait to check if there are offers coming from the
preferred slaves. Of course, there should be a timeout mechanism that data
locality can be relaxed if no offers are from the preferred slaves in a
reasonable time period.
> [MESOS] Improve data locality when launching new executors when dynamic
> allocation is enabled
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-16944
> URL: https://issues.apache.org/jira/browse/SPARK-16944
> Project: Spark
> Issue Type: New Feature
> Components: Mesos
> Affects Versions: 2.0.0
> Reporter: Sun Rui
>
> Currently Spark on Yarn supports better data locality by considering the
> preferred locations of the pending tasks when dynamic allocation is enabled.
> Refer to https://issues.apache.org/jira/browse/SPARK-4352. It would be better
> that Mesos can also support this feature.
> I guess that some logic existing in Yarn could be reused by Mesos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
