[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21577 > We still need the solution proposed in #21558 for the v2 API Should we have a separate bug for these then? I just piggybacked on the bug you filed, but if they're separate issues, even if c

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21577 +1. This fixes the commit coordinator problem where two separate tasks can be authorized. That case could lead to duplicate data (if, for example, both tasks generated unique file names using a rando

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92039/ Test FAILed. ---

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21577 **[Test build #92039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92039/testReport)** for PR 21577 at commit [`a2f4c1b`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21577 **[Test build #92039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92039/testReport)** for PR 21577 at commit [`a2f4c1b`](https://github.com/apache/spark/commit/a2

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4179/ Tes

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/285/

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21577 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21577: [SPARK-24552][core] Correctly identify tasks in output c...

2018-06-18 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21577 I tried to create a test based on actually running a job, but I'd have to do a lot of hacking to control what the result stage does, and it was starting to feel not much better than the unit test I a