Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1934#discussion_r16337225
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -213,27 +213,21 @@ class ApplicationMaster(args:
ApplicationMasterArguments, conf: Configuration,
ApplicationMaster.sparkContextRef.wait(waitTime)
}
sparkContext = ApplicationMaster.sparkContextRef.get()
- assert(sparkContext != null || numTries >= maxNumTries)
- if (sparkContext != null) {
+ if (sparkContext == null) {
+ logError(("Unable to retrieve SparkContext in spite of waiting
for %d, maxNumTries = %d."
--- End diff --
This language is quite specific to the internal implementation and might be
a bit confusing for users. For instance, what does it mean to "retrieve" a
SparkContext? It might be better to make it more consistent with the language
in the two other log statements and say something like:
```
Spark context did not initialize properly after waiting for XXX.
Please check earlier log output for errors.
Failing the application.
```
Also - I don't think this will print a unit for the wait time. I believe
the unit is `ms` so it might be good to add that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
