GitHub user CodingCat opened a pull request:
https://github.com/apache/spark/pull/8610
[SPARK-5337][Mesos][Standalone] respect spark.task.cpus when scheduling
Applications
https://issues.apache.org/jira/browse/SPARK-5337
Currently, we didn't consider spark.task.cpus when scheduling the
applications in Master, so that we may fall into one of the following cases
the executor gets N cores but we need M cores to run a single task, where N
< M
the executor gets N cores, we need M cores to run a single task, where N %
M != 0 && N > M; so that we waste some cores in the executor
Patch for YARN is in submitted by @WangTaoTheTonic : #4123
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/CodingCat/spark SPARK-5337-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8610.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8610
----
commit d3289fc331bd2e6bb19453f3489ff3ca4d865c90
Author: CodingCat <[email protected]>
Date: 2015-09-05T06:01:05Z
respect spark.task.cpus when scheduling Applications
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
