[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20915 Ah, I am happy too that it's added back. Was just simply wondering if I can fix the fix version in SPARK-12850 JIRA. Nothing more then this. Thank you @cloud-fan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20915 IIUC the bucket pruning was accidentally removed during refactoring, I'm happy to see it's added back. The migration must keep all the existing features. I think we have enough tests to guarantee it now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20915 @cloud-fan, how does it relate to SPARK-23803, SPARK-12850 and SPARK-23507? Was about to take an action to the JIRAs but felt better making sure ahead. SPARK-12850 was merged in 2.0.0 but reverted by SPARK-14535 in 2.0.0 so it's no problem but SPARK-23803 duplicates SPARK-12850. and .. you plan to migrate file-based source to datasource v2 which includes refactoring this feature? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20915 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91500/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91500/testReport)** for PR 20915 at commit [`f949128`](https://github.com/apache/spark/commit/f94912896c1859d078fc52fda867beaecaa68a47). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91501/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91501/testReport)** for PR 20915 at commit [`72712c9`](https://github.com/apache/spark/commit/72712c951338d3acbd1e141717d779ed33b03d82). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91501/testReport)** for PR 20915 at commit [`72712c9`](https://github.com/apache/spark/commit/72712c951338d3acbd1e141717d779ed33b03d82). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user sabanas commented on the issue: https://github.com/apache/spark/pull/20915 @cloud-fan thanks for the feedback, I fixed accordingly. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91500/testReport)** for PR 20915 at commit [`f949128`](https://github.com/apache/spark/commit/f94912896c1859d078fc52fda867beaecaa68a47). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91492/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91492/testReport)** for PR 20915 at commit [`de9ecb6`](https://github.com/apache/spark/commit/de9ecb64068d1cafa79ce61a2bf1795981a46f46). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #91492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91492/testReport)** for PR 20915 at commit [`de9ecb6`](https://github.com/apache/spark/commit/de9ecb64068d1cafa79ce61a2bf1795981a46f46). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20915 @sabanas @pwoody Sorry for the late reply. We are migrating Parquet, ORC and the other data sources using Data Source API V2. Thus, we are trying to address all these issues in the new code. @cloud-fan and @gengliangwang are leading the migration efforts now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user sabanas commented on the issue: https://github.com/apache/spark/pull/20915 ping @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user sabanas commented on the issue: https://github.com/apache/spark/pull/20915 @gatorsmile we would really want to start using this feature in our production stack. could you or anyone else give your feedback on this PR? Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/20915 Ping @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/20915 @gatorsmile any chance you could take a look at this? Let me know if there is someone more appropriate to review, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/20915 @gatorsmile would appreciate your feedback on this PR. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user sabanas commented on the issue: https://github.com/apache/spark/pull/20915 @gatorsmile can we please proceed with the review? thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88753/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88753 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88753/testReport)** for PR 20915 at commit [`697784d`](https://github.com/apache/spark/commit/697784d57642ffe2bda55e518a34265829e4ba2c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88753/testReport)** for PR 20915 at commit [`697784d`](https://github.com/apache/spark/commit/697784d57642ffe2bda55e518a34265829e4ba2c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88706/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88706/testReport)** for PR 20915 at commit [`8f6bc28`](https://github.com/apache/spark/commit/8f6bc28752a5ef59d8914f4b37e9e47339feb04f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88706/testReport)** for PR 20915 at commit [`8f6bc28`](https://github.com/apache/spark/commit/8f6bc28752a5ef59d8914f4b37e9e47339feb04f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88698/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20915 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88698/testReport)** for PR 20915 at commit [`cb36012`](https://github.com/apache/spark/commit/cb36012f8aadc9e6c842e1ff143394dd5de71be5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20915 **[Test build #88698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88698/testReport)** for PR 20915 at commit [`cb36012`](https://github.com/apache/spark/commit/cb36012f8aadc9e6c842e1ff143394dd5de71be5). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20915 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20915: [SPARK-23803][SQL] Support bucket pruning
Github user sabanas commented on the issue: https://github.com/apache/spark/pull/20915 org.apache.spark.sql.execution.streaming.sources.RateSourceSuite.overflow test failed. it passes on my local dev env and there are several other PRs failing the same test. possible CI issues? can we run it again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
