Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12994
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218184446
+1 Thanks @jerryshao
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218177238
I'm not concerned with the downgrade case. It just won't find the file if
yarn isn't setting the recovery path any longer (it will create new one in
localdir) , but
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218074876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218074797
**[Test build #58216 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58216/consoleFull)**
for PR 12994 at commit
[`6d4a8f1`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218074874
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218072671
**[Test build #58216 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58216/consoleFull)**
for PR 12994 at commit
[`6d4a8f1`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218066986
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218066984
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218066972
**[Test build #58210 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58210/consoleFull)**
for PR 12994 at commit
[`02752c9`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218065383
**[Test build #58210 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58210/consoleFull)**
for PR 12994 at commit
[`02752c9`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218064524
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218064520
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218064500
**[Test build #58207 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58207/consoleFull)**
for PR 12994 at commit
[`519bf07`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218062949
**[Test build #58207 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58207/consoleFull)**
for PR 12994 at commit
[`519bf07`](https://gi
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218062741
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218060958
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218060959
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218060952
**[Test build #58205 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58205/consoleFull)**
for PR 12994 at commit
[`519bf07`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218060219
**[Test build #58205 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58205/consoleFull)**
for PR 12994 at commit
[`519bf07`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218051051
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218051038
**[Test build #58201 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58201/consoleFull)**
for PR 12994 at commit
[`4e5c2fd`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218051052
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218050017
@tgravescs , I tested locally using Hadoop 2.4 and 2.6 with different
scenarios:
1. Only Hadoop 2.4
2. Hadoop 2.4 upgrade to 2.6 with NM recovery disabled
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218049583
**[Test build #58201 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58201/consoleFull)**
for PR 12994 at commit
[`4e5c2fd`](https://gi
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-218034727
Thanks @tgravescs for your comments, I will change the code and do a more
comprehensive test accordingly.
---
If your project is set up for it, you can reply to this
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-217873160
few minor comments but mostly looks good. Did you build against both
hadoop 2.5+ and hadoop < 2.5?
Did you manually test the upgrade path?
---
If your proj
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/12994#discussion_r62502165
--- Diff:
yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala
---
@@ -234,7 +236,25 @@ class YarnShuffleServiceSuite extends S
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/12994#discussion_r62500518
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -222,4 +219,43 @@ protected void serviceStop() {
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/12994#discussion_r62500214
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -112,14 +118,15 @@ private boolean isAuthenticati
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/12994#discussion_r62493162
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -222,4 +219,43 @@ protected void serviceStop() {
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-217764236
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-217764237
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-217764184
**[Test build #58116 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58116/consoleFull)**
for PR 12994 at commit
[`08557bf`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12994#issuecomment-217763069
**[Test build #58116 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58116/consoleFull)**
for PR 12994 at commit
[`08557bf`](https://gi
GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/12994
[SPARK-14963][Yarn] Using recoveryPath if NM recovery is enabled
## What changes were proposed in this pull request?
From Hadoop 2.5+, Yarn NM supports NM recovery which using recovery pa
36 matches
Mail list logo
