[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-89023289 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-89023264 [Test build #29620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29620/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-89140693 @JoshRosen, no problem. I'm happy to contribute. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-88995927 @JoshRosen, thanks for the help. I just rebased this PR against master. It should also fix the conflict. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-88996367 [Test build #29620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29620/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5124 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-89047277 I've merged this into `master` (1.4.0), `branch-1.3` (1.3.1), and `branch-1.2` (1.2.2). Thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87864772 Aside from two minor comments that I just left, this looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27439228 --- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala --- @@ -173,4 +176,21 @@ class SparkContextSuite extends FunSuite with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27439385 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -675,7 +675,7 @@ class DAGScheduler( // Cancel all jobs

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27448459 --- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala --- @@ -173,4 +176,21 @@ class SparkContextSuite extends FunSuite with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87902682 [Test build #29439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29439/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27448572 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -675,7 +675,7 @@ class DAGScheduler( // Cancel all jobs belonging

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87925072 [Test build #29439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29439/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87925087 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27307726 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27308927 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27309179 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27315656 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27305215 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27309136 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -504,7 +504,7 @@ class DAGScheduler( callSite: CallSite,

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27306885 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87143838 [Test build #29335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29335/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87142529 [Test build #29334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29334/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87153979 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87153978 [Test build #29335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29335/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87142944 @JoshRosen I updated PR, please review. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87153677 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-87153674 [Test build #29334 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29334/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27276816 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r27276853 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-86768612 [Test build #29266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29266/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-86769103 @JoshRosen , thanks for the help. I updated this PR with 1. regression test 2. use Option[Properties] for the value of SparkContext.localProperties 3. add

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-86787830 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-86787823 [Test build #29266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29266/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r26977523 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -805,7 +806,7 @@ class DAGScheduler( } val

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85157364 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85157322 [Test build #29009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29009/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85186794 @JoshRosen thanks for the suggestions. Let's me work on those tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r26961357 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -805,7 +806,7 @@ class DAGScheduler( } val

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85132931 (Sorry, that should have been `SparkContext.localProperties.initialValue` above; I've revised my comment) --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85124958 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85125446 [Test build #29009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29009/consoleFull) for PR 5124 at commit

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85132588 It looks like this NPE bug has been around for a while, but it seems pretty hard to hit (which is probably why it hasn't been reported before). I think that we should

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-21 Thread hunglin
GitHub user hunglin opened a pull request: https://github.com/apache/spark/pull/5124 SPARK-6414: Spark driver failed with NPE on job cancelation Use Option for ActiveJob.properties to avoid NPE bug You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-84525564 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-21 Thread hunglin
Github user hunglin commented on a diff in the pull request: https://github.com/apache/spark/pull/5124#discussion_r26900122 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -805,7 +806,7 @@ class DAGScheduler( } val