[
https://issues.apache.org/jira/browse/SPARK-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411319#comment-15411319
]
Sun Rui edited comment on SPARK-16944 at 8/8/16 5:29 AM:
---------------------------------------------------------
I don't think it can be improved without dynamic allocation. because without
dynamic allocation, you can't know the nodes where the data reside in advance.
But it can be compensated by allocating at least one executor on each salve,
but may be some waste of executors.
was (Author: sunrui):
I don't think it can be improved without dynamic allocation. because without
dynamic allocation, you can't know the nodes where the data reside in advance.
But it can be compensated by allocating at least one executor on salves, but
may be some waste of executors.
> [MESOS] Improve data locality when launching new executors when dynamic
> allocation is enabled
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-16944
> URL: https://issues.apache.org/jira/browse/SPARK-16944
> Project: Spark
> Issue Type: New Feature
> Components: Mesos
> Affects Versions: 2.0.0
> Reporter: Sun Rui
>
> Currently Spark on Yarn supports better data locality by considering the
> preferred locations of the pending tasks when dynamic allocation is enabled.
> Refer to https://issues.apache.org/jira/browse/SPARK-4352. It would be better
> that Mesos can also support this feature.
> I guess that some logic existing in Yarn could be reused by Mesos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
