[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15923 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15923 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3481/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3481 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3481/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3480 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3480/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3475/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3475 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3475/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69825/ Test FAILed. ---

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69825/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69825/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-07 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15923 @JoshRosen Added a test for `detectCorrupt` is false. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-06 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15923 This looks good overall, but one nit: it looks like we don't have any test coverage for the case where `detectCorrupt` is false. We should probably add a test to make sure that the feature flag

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-06 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15923 ping @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-02 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15923 cc @zsxwing @JoshRosen does this look good? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3448/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3448 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3448/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69422/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69422/ Test FAILed. ---

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69422/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3444 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3444/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #3443 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3443/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69261/ Test FAILed. ---

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69261/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #69261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69261/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-28 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15923 Manually test this patch with a job that usually failed because of corrupt stream, as the logging said: ``` 16/11/20 08:32:07 WARN ShuffleBlockFetcherIterator: got an corrupted block

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15923 @joshrosen @zsxwing Could you help to review this one ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68870/ Test PASSed. ---

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #68870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68870/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #68870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68870/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68813/ Test FAILed. ---

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #68813 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68813/consoleFull)** for PR 15923 at commit

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15923 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15923 **[Test build #68813 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68813/consoleFull)** for PR 15923 at commit