Github user henryr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21302#discussion_r188042296
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -602,6 +602,16 @@ class ParquetFilterSuite extends QueryTest with
ParquetTest with SharedSQLContex
}
}
}
+
+ test("SPARK-23852: Broken Parquet push-down for partially-written
stats") {
+ // parquet-1217.parquet contains a single column with values -1, 0, 1,
2 and null.
+ // The row-group statistics include null counts, but not min and max
values, which
+ // triggers PARQUET-1217.
+ val df = readResourceParquetFile("test-data/parquet-1217.parquet")
--- End diff --
PR for master is https://github.com/apache/spark/pull/21323. My guess is
there's no reason to block this backport and 2.3.1 by waiting for it to land,
but happy to do whatever.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
