[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97771/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97749/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97749/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97698/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97701/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97701/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97698/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4227/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97701/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97698/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4166/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/22561 @cloud-fan Yes, it has problems for Not expression, we need find some good ways. Thanks for review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22561 Your rewrite is applied to the entire predicate expression tree, how about the case `NOT(p AND x)`? If `p` is true, `x` is false, then `NOT(p AND x)` is true. But `NOT(true AND true)` is false. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/22561 @cloud-fan which should be proved is that: the partitions returned of p' shound contain the partitions returned by p. Here, let p' = p && x, if x is true then p' == p; else if x is false, the paritions returned we needed is none, so p' contains the partitions we needed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22561 Please describe the idea clearly in the PR description. After reading the code, I think the idea is, for any predicate `p`, we can rewrite it to `p'` that only contains partition columns. The requirement is, partitions returned by the partition predicate `p`, must also be returned by `p'`. We need to be careful here, to not introduce correctness issues. Can you prove that your rewrite is safe? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97373/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97373/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #97373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97373/testReport)** for PR 22561 at commit [`a13fe23`](https://github.com/apache/spark/commit/a13fe239ec2f44cdb09a5b66d50e018b55dc8ee6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3968/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96678/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96678/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3526/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96678/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22561 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96668/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96668/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3518/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96668/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/22561 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96654/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96654 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96654/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user eatoncys commented on the issue: https://github.com/apache/spark/pull/22561 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22561 **[Test build #96654 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96654/testReport)** for PR 22561 at commit [`6acb460`](https://github.com/apache/spark/commit/6acb460381c96fe71f807f94bb617f3928f41694). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22561: [SPARK-25548][SQL]In the PruneFileSourcePartitions optim...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3507/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
