[
https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896192#comment-15896192
]
Sean Owen commented on SPARK-19823:
-----------------------------------
I don't think that's quite right. Task assignment takes into account locality,
for example. Choosing to override locality preferences of the scheduler and
batch-assign tasks to executors that are suboptimal for locality has drawbacks.
You will generally observe that executors that are local to data that jobs need
will get more tasks, and those that aren't will already be less busy, and
therefore more quickly become idle and terminate. The effect you want already
takes place indirectly.
> Support Gang Distribution of Task
> ----------------------------------
>
> Key: SPARK-19823
> URL: https://issues.apache.org/jira/browse/SPARK-19823
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler, Spark Core
> Affects Versions: 2.0.2
> Reporter: DjvuLee
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
