[
https://issues.apache.org/jira/browse/SPARK-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995922#comment-14995922
]
Hyukjin Kwon edited comment on SPARK-9557 at 11/9/15 2:24 AM:
--------------------------------------------------------------
But I would like to just add a just little bit of corrections.
I updated the test for this by not reducing just collecting filters as an
array; however, this actually creates filters after reducing in the
{{DataSourceStrategy}}, which the original test codes followed.
was (Author: hyukjin.kwon):
But I would like to just add a just little corrections.
I update the test for this by not reducing just collecting filters as an array;
however, this actually create filters after reducing in the
{{DataSourceStrategy}}, which the original test codes followed this.
> Refactor ParquetFilterSuite and remove old ParquetFilters code
> --------------------------------------------------------------
>
> Key: SPARK-9557
> URL: https://issues.apache.org/jira/browse/SPARK-9557
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> Currently there are two Parquet filter conversion code path, one for
> converting data sources {{Filter}}, the other for converting Catalyst
> predicate {{Expression}}, which is used by the removed old Parquet code. We
> should remove the latter, but {{ParquetFilterSuite}} uses it to test Parquet
> filter push-down.
> Need to refactor {{ParquetFilterSuite}} to make it test the data source
> version and then remove the old Parquet filter conversion code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
