GitHub user berngp opened a pull request:
https://github.com/apache/spark/pull/433
[SPARK-1522] : YARN ClientBase throws a NPE if there is no YARN applicat...
...ion specific CP
The current implementation of
`ClientBase.getDefaultYarnApplicationClasspath` inspects
the `MRJobConfig` class for the field `DEFAULT_YARN_APPLICATION_CLASSPATH`
when it should
be really looking into `YarnConfiguration`. If the Application
Configuration has no
`yarn.application.classpath` defined a NPE exception will be thrown.
Additional Changes include:
* ScalaBuild now points to Scalatest 2.1.3
* ScalaBuild project "root" renamed as "spark"
* Test Suite for ClientBase added
* Fixes for scalastyle in other yarn files.
[ticket: SPARK-1522] : https://issues.apache.org/jira/browse/SPARK-1522
Author : [email protected]
Testing : SPARK_HADOOP_VERSION=2.3.0 SPARK_YARN=true ./sbt/sbt test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/berngp/spark feature/SPARK-1522
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/433.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #433
----
commit b08beb96c4437f095c31f53a0877eb95aebab637
Author: Bernardo Gomez Palacio <[email protected]>
Date: 2014-04-17T09:35:52Z
[SPARK-1522] : YARN ClientBase throws a NPE if there is no YARN application
specific CP
The current implementation of
`ClientBase.getDefaultYarnApplicationClasspath` inspects
the `MRJobConfig` class for the field `DEFAULT_YARN_APPLICATION_CLASSPATH`
when it should
be really looking into `YarnConfiguration`. If the Application
Configuration has no
`yarn.application.classpath` defined a NPE exception will be thrown.
Additional Changes include:
* ScalaBuild now points to Scalatest 2.1.3
* ScalaBuild project "root" renamed as "spark"
* Test Suite for ClientBase added
* Fixes for scalastyle in other yarn files.
[ticket: SPARK-1522] : https://issues.apache.org/jira/browse/SPARK-1522
Author : [email protected]
Testing : SPARK_HADOOP_VERSION=2.3.0 SPARK_YARN=true ./sbt/sbt test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
