[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 Sure. Although I'm not sure why other committers who review PRs don't just merge them too... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20893 I see. Could you merge that PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 The test is fixed by #20950. Just push that one instead. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20893 Let me revert this PR from the master. Please resubmit the PR after addressing the flaky tests. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20893 cc @vanzin @sahilTakiar @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20893 Could we revert the PR from the master brach too? https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.7/4303/testReport/org.apache.spark.launcher/LauncherServerSuite/testAppHandleDisconnect/history/ --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20893 Thank you so much for swift action, @vanzin ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 Let me revert the change from 2.3 and open a bug for the flaky test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20893 Hi, @vanzin and @sahilTakiar . Could you take a look at `branch-2.3` hadoop-2.6 parts? It dosen't happen for all branches, but it seems to become unstable a little bit. - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.3-test-maven-hadoop-2.6/311/testReport/junit/org.apache.spark.launcher/LauncherServerSuite/testAppHandleDisconnect/history/ (3 serial failures) - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.3-test-sbt-hadoop-2.6/278/ (1 failure) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 Merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88679/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88679 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88679/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88679/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 SGTM. Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user sahilTakiar commented on the issue:
https://github.com/apache/spark/pull/20893
Wrote a test in `SparkLauncherSuite` and was able to replicate the error
from HIVE-18533, and then realized the exception is only logged and then
swallowed. From `SparkContext`
```
if (_dagScheduler != null) {
Utils.tryLogNonFatalError {
_dagScheduler.stop()
}
_dagScheduler = null
}
```
Seems the failures I was seeing in HIVE-18533 are due to something else.
Regardless, this is still probably a good fix since you still don't want to
write to the connection unless its open, but given that the exception is only
logged and not thrown, don't see an easy way to write a test for this.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88604/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88604/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user sahilTakiar commented on the issue: https://github.com/apache/spark/pull/20893 Ok, I'll work on writing a test for `SparkLauncherSuite`. The test added here was meant to cover the race condition mentioned [here|https://spark.apache.org/docs/2.3.0/sql-programming-guide.html#bucketing-sorting-and-partitioning] --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 The change looks ok but the test being added is totally unrelated to it. If you want to write a test, it should be not too hard to add something to `SparkLauncherSuite` in `core/`. Start an in-process app that waits on a lock, disconnect the handle, then wake up the lock so the app finishes successfully. That should fail without your fix and work with it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88604/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20893 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88577/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88577/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88577/testReport)** for PR 20893 at commit [`4ca8a32`](https://github.com/apache/spark/commit/4ca8a32e2a518f3c7ccecd406a8b03eac06f860b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20893 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20893 The change looks good, cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
