[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 backported to 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90703/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90703/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90703/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90629/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90629/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90629/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90623/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90623/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90623/testReport)** for PR 21114 at commit [`1b1b1fa`](https://github.com/apache/spark/commit/1b1b1fa6c8c89759c4353c843fb67282c2baac6f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 There's "get accum" test which does this, it was updated for new behavior. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 can we do this? ``` var acc = ... ... // launch a long running job val accId = acc.getId acc = null gc ... // job finished ``` accumulator is created by users so we have to be prepared for any situations. That's why we use weak reference at the first place. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90577/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90577/testReport)** for PR 21114 at commit [`8b30733`](https://github.com/apache/spark/commit/8b30733dba85d9881d0171414616bd0b0893f419). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 Just a long-running job and memory pressure is not enough. You need to have several attempts for a stage, each new attempt will update Stage._latestInfo, so previous StageInfo and it's accumulators can be GCed. After this AccumulatorContext.get() throws an exception until GCed accumulators are removed by ContextCleaner. It's also important to send an accumulator update for an old attempt before all tasks are finished, otherwise the stage will be marked as completed, removed from DAGScheduler.stageIdToStage and DAGScheduler.handleTaskSetFailed() will be ignored. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 I may miss something. It's weak reference, so if we have a long-running job and the memory pressure is high, we can reproduce it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 This issue is more like a race condition, so the test needs to generate a specific sequence of events to reproduce the issue. I agree it's probably too specific. What is Spark approach to reproduce this kind of issues? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21114 **[Test build #90577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90577/testReport)** for PR 21114 at commit [`8b30733`](https://github.com/apache/spark/commit/8b30733dba85d9881d0171414616bd0b0893f419). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 this behavior change LGTM, but the test is over complicated and seems has limited value. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21114 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 Yes, this is correct. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21114 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
