GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/8100
[SPARK-9817][YARN] Improve the locality calculation of containers by taking
pending container requests into consideraion
This is a follow-up PR to further improve the locality calculation by
considering the pending container's request. Since the locality preferences of
tasks may be shifted from time to time, current localities of pending container
requests may not fully match the new preferences, this PR improve it by
removing outdated, unmatched container requests and replace with new requests.
@sryza please help to review, thanks a lot.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-9817
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8100.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8100
----
commit c4a6a5798408158470ea6694366a60b96f1a5ea7
Author: jerryshao <[email protected]>
Date: 2015-08-11T09:13:15Z
Improve the locality placement strategy by considering the localities of
pending container requests
commit 3f36e2fdaaf0926da3edd9a7796fe39636c21b46
Author: jerryshao <[email protected]>
Date: 2015-08-11T09:42:33Z
Fix the docs
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
