[
https://issues.apache.org/jira/browse/PARQUET-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue updated PARQUET-1309:
-------------------------------
Description: In SPARK-24251, we found that the changes to use
HadoopReadOptions accidentally switched the [properties that enable stats and
dictionary
filters|https://github.com/apache/parquet-mr/blob/8bbc6cb95fd9b4b9e86c924ca1e40fd555ecac1d/parquet-hadoop/src/main/java/org/apache/parquet/HadoopReadOptions.java#L83].
Both are enabled by default so it is unlikely that anyone will need to turn
them off and there is an easy work-around, but we should fix the properties for
1.10.1. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on
1.8.x). (was: In SPARK-24251, we found that the changes to use
HadoopReadOptions accidentally switched the [properties that enable stats and
dictionary
filters|https://github.com/apache/parquet-mr/blob/8bbc6cb95fd9b4b9e86c924ca1e40fd555ecac1d/parquet-hadoop/src/main/java/org/apache/parquet/HadoopReadOptions.java#L83].
Both are enabled by default so it is unlikely that anyone will need to turn
them off and there is an easy work-around, but we should fix the properties for
1.10.0. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on
1.8.x).)
> Parquet Java uses incorrect stats and dictionary filter properties
> ------------------------------------------------------------------
>
> Key: PARQUET-1309
> URL: https://issues.apache.org/jira/browse/PARQUET-1309
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Reporter: Ryan Blue
> Priority: Major
> Fix For: 1.10.1
>
>
> In SPARK-24251, we found that the changes to use HadoopReadOptions
> accidentally switched the [properties that enable stats and dictionary
> filters|https://github.com/apache/parquet-mr/blob/8bbc6cb95fd9b4b9e86c924ca1e40fd555ecac1d/parquet-hadoop/src/main/java/org/apache/parquet/HadoopReadOptions.java#L83].
> Both are enabled by default so it is unlikely that anyone will need to turn
> them off and there is an easy work-around, but we should fix the properties
> for 1.10.1. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is
> on 1.8.x).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
