[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21696 Thanks, @wangyum! I think this is refactor was a good idea. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92603/ Test PASSed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92603/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92599/ Test PASSed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92599/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92600/ Test PASSed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92600/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92603/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/667/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92600/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/664/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92599/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92595/ Test FAILed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92595/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92595/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/660/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Don't block on me. Just wanted to doubly sure if this is the only way. I am fine. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Yea, I got that we it should have been done like this, and wonder if we can avoid this. It sounds more like a bandaid fix mainly cased by decimal. FWIW, in case of timestamp as INT96

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 you can't get the physical schema information in a higher layer, as it may vary for different files. The table schema can evolve(add/drop column). ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 physical schema information shouldn't usually referred in a higher layer though cc @liancheng. It's kind of something we should avoid .. I got that we need this but wonder if this is the only

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 makes sense to me, since we need the physical schema information to pushdown decimal and timestamp. also cc @rdblue @michal-databricks ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92529/ Test PASSed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92529/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92530/ Test FAILed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92530/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92523/ Test PASSed. ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92530/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/623/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/622/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92529/testReport)** for PR 21696 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/616/

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)** for PR 21696 at commit