[
https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096
]
DjvuLee commented on SPARK-19823:
---------------------------------
When Spark distributes tasks to Executors, it uses a Round-Robin way, this
means we distribute tasks to all Executors as possible, this is a good strategy
in most case.
However, when we introduce the dynamic allocation and concurrent jobs, users
may need the gang distribution for one single job, this means we allocate tasks
for one job in few Executors, so we can release the unused resource.
In current ways, Executors may not get released since the long-running stage
will distributed on task in each Executors, even we have more cores in each
Executor.
We can offer a configuration for users, and this will not introduce much
complexity.
> Support Gang Distribution of Task
> ----------------------------------
>
> Key: SPARK-19823
> URL: https://issues.apache.org/jira/browse/SPARK-19823
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler, Spark Core
> Affects Versions: 2.0.2
> Reporter: DjvuLee
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
