nastra commented on code in PR #3607:
URL: https://github.com/apache/parquet-java/pull/3607#discussion_r3628569619
##########
parquet-hadoop/src/test/java/org/apache/parquet/format/converter/TestParquetMetadataConverter.java:
##########
@@ -2159,4 +2159,110 @@ public void testColumnIndexNanCountsRoundTrip() {
assertNotNull(roundTrip);
assertEquals(List.of(1L, 0L, 0L), roundTrip.getNanCounts());
}
+
+ @Test
+ public void testCorruptStatsPerColumnGate() {
+ // A created_by string from a version known to have the PARQUET-251 bug
+ String corruptCreatedBy = "parquet-mr version 1.6.0 (build abcd)";
+
+ // Set up legacy V1 statistics with min/max (not min_value/max_value)
+ org.apache.parquet.format.Statistics formatStats = new
org.apache.parquet.format.Statistics();
+ byte[] minBytes = new byte[] {0, 1, 2, 3};
+ byte[] maxBytes = new byte[] {4, 5, 6, 7};
+ formatStats.setMin(minBytes);
+ formatStats.setMax(maxBytes);
+ formatStats.setNull_count(5);
+
+ // For BINARY column: stats should be ignored (only null_count preserved)
+ PrimitiveType binaryType = new PrimitiveType(Repetition.OPTIONAL,
PrimitiveTypeName.BINARY, "bin_col");
+ Statistics binaryStats =
ParquetMetadataConverter.fromParquetStatisticsInternal(
+ corruptCreatedBy, formatStats, binaryType,
ParquetMetadataConverter.SortOrder.SIGNED);
+ assertFalse("BINARY min/max should be ignored for corrupt file",
binaryStats.hasNonNullValue());
Review Comment:
you might need to rebase the PR because we switched all assertions to
AssertJ and removed all JUnit4 usages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
