[
https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386769#comment-16386769
]
Apache Spark commented on SPARK-23604:
--------------------------------------
User 'henryr' has created a pull request for this issue:
https://github.com/apache/spark/pull/20740
> ParquetInteroperabilityTest timestamp test should use
> Statistics.hasNonNullValue
> --------------------------------------------------------------------------------
>
> Key: SPARK-23604
> URL: https://issues.apache.org/jira/browse/SPARK-23604
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Henry Robinson
> Priority: Minor
>
> We ran into an issue with a downstream build of Spark running against a
> custom Parquet build where {{ParquetInteroperabilityTestSuite}} started
> failing because {{Statistics.isEmpty}} changed its behavior as of
> PARQUET-1217. ({{isEmpty() now considers whether there are 0 or more nulls,
> and by default {{num_nulls}} is 0 for 'empty' stats objects).
> The test really cares about whether the statistics object has values, so a
> very simple fix to use {{hasNonNullValue}} instead corrects the issue. Filing
> it now because it's a backwards-compatible fix to the current Parquet version
> so we can fix it right now before we hit the issue in the future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
