[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21696 Thanks, @wangyum! I think this is refactor was a good idea. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92603/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92603/testReport)** for PR 21696 at commit [`9e1aa9d`](https://github.com/apache/spark/commit/9e1aa9d06f9a07b6f73f38ce264c54a7e309ad94). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92599/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92599/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92600/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92600/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92603/testReport)** for PR 21696 at commit [`9e1aa9d`](https://github.com/apache/spark/commit/9e1aa9d06f9a07b6f73f38ce264c54a7e309ad94). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/667/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92600/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/664/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92599/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92595/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92595/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92595/testReport)** for PR 21696 at commit [`9b489ec`](https://github.com/apache/spark/commit/9b489ecf732d2e8a455d1d7ba5fd96a17295292c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/660/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Don't block on me. Just wanted to doubly sure if this is the only way. I am fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Yea, I got that we it should have been done like this, and wonder if we can avoid this. It sounds more like a bandaid fix mainly cased by decimal. FWIW, in case of timestamp as INT96 (deprecated in Parquet), this is a legacy and something we should remove out. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 you can't get the physical schema information in a higher layer, as it may vary for different files. The table schema can evolve(add/drop column). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 physical schema information shouldn't usually referred in a higher layer though cc @liancheng. It's kind of something we should avoid .. I got that we need this but wonder if this is the only way to get through. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 makes sense to me, since we need the physical schema information to pushdown decimal and timestamp. also cc @rdblue @michal-databricks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92529/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92529/testReport)** for PR 21696 at commit [`e7b4ec9`](https://github.com/apache/spark/commit/e7b4ec9c7656dd5d5bf1b570a65035dfea91dc07). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92530/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92530/testReport)** for PR 21696 at commit [`07dd372`](https://github.com/apache/spark/commit/07dd372f2f12d788b9ed3452ade02cb38999ad03). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92523/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)** for PR 21696 at commit [`10d408f`](https://github.com/apache/spark/commit/10d408fd3fe429d5529755b5abebac86b22b6d55). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` case class ParquetSchemaType(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92530/testReport)** for PR 21696 at commit [`07dd372`](https://github.com/apache/spark/commit/07dd372f2f12d788b9ed3452ade02cb38999ad03). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/623/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/622/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92529/testReport)** for PR 21696 at commit [`e7b4ec9`](https://github.com/apache/spark/commit/e7b4ec9c7656dd5d5bf1b570a65035dfea91dc07). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/616/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)** for PR 21696 at commit [`10d408f`](https://github.com/apache/spark/commit/10d408fd3fe429d5529755b5abebac86b22b6d55). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
