rdblue commented on code in PR #3610:
URL: https://github.com/apache/parquet-java/pull/3610#discussion_r3562468870
##########
parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java:
##########
@@ -2057,7 +2064,17 @@ private void buildChildren(
|| schemaElement.converted_type == ConvertedType.INTERVAL)) {
columnOrder = org.apache.parquet.schema.ColumnOrder.undefined();
}
+ // INT96_TIMESTAMP_ORDER is only valid for INT96 columns, ignore it
anywhere else.
+ if (columnOrder.getColumnOrderName() ==
ColumnOrderName.INT96_TIMESTAMP_ORDER
+ && schemaElement.type != Type.INT96) {
+ columnOrder = org.apache.parquet.schema.ColumnOrder.undefined();
+ }
primitiveBuilder.columnOrder(columnOrder);
+ } else if (schemaElement.type == Type.INT96) {
+ // A footer without column orders predates INT96_TIMESTAMP_ORDER, so
an INT96 column here
+ // must not inherit the (chronological) construction-time default:
its stats, if any, were
+ // written under the legacy order and must be ignored.
+
primitiveBuilder.columnOrder(org.apache.parquet.schema.ColumnOrder.undefined());
Review Comment:
There isn't a test in `TestParquetMetadataConverter` that validates Parquet
metadata for an INT96 without a column order. There is a test that a new schema
will produce a column order, but not one to verify that a schema without one
will be read correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
