GitHub user subrotosanyal opened a pull request:
https://github.com/apache/spark/pull/13658
[SPARK-15937] [yarn] Improving the logic to wait for an initialised Spark
Context
## What changes were proposed in this pull request?
As per this fix we take into account if the job has already finished while
waiting instead of just basing the logic if the Spark Context reference is
available or not. Similar approach is being is used in
org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(securityMgr:
SecurityManager) to check if the job is finished or not before declaring it
failed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/subrotosanyal/spark
SPARK-15937-fix-wait-for-sparkContext-logic
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13658.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13658
----
commit 66351191cae541faaa5850a72d1778b0fdb7eca1
Author: Subroto Sanyal <[email protected]>
Date: 2016-06-14T08:04:34Z
SPARK-15937 The method waitForSparkContextInitialized() and runDriver()
will taken into account if the job has been finished or not already before
declaring the job to be failed.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
