GitHub user eatoncys opened a pull request:
https://github.com/apache/spark/pull/18322
[SPARK-21115][Core]If the cores left is less than the coresPerExecutor,the
cores left will not be allocated, so it should not to check in every schedule
## What changes were proposed in this pull request?
If we start an app with the param --total-executor-cores=4 and
spark.executor.cores=3, the cores left is always 1, so it will try to allocate
executors in the function
org.apache.spark.deploy.master.startExecutorsOnWorkers in every schedule.
Another question is, is it will be better to allocate another executor with
1 core for the cores left.
## How was this patch tested?
unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/eatoncys/spark leftcores
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18322.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18322
----
commit 5f5f01fb55ec827d0053de2b574929520ff73406
Author: 10129659 <[email protected]>
Date: 2017-06-16T05:46:47Z
If the cores left is less than the coresPerExecutor,the cores left will not
be allocated, so it should not to check in every schedule
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
