GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/7786
[SPARK-9468][Yarn][Core] Avoid scheduling tasks on preemption executors
For Spark running on Yarn, if preemption is enabled by Yarn, containers
will be preempted by Yarn, this will be notified to AM ahead of time. Spark
itself should be aware of this preemption information and avoid scheduling
tasks on these executors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-9468
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7786.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7786
----
commit 6d030ec8a9b1e616c2c7c17697de7c5f703bd2af
Author: jerryshao <[email protected]>
Date: 2015-07-30T05:56:17Z
Add preemption mechanism for Spark on Yarn
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
