rdblue commented on a change in pull request #1335:
URL: https://github.com/apache/iceberg/pull/1335#discussion_r470256559
##########
File path: parquet/src/main/java/org/apache/iceberg/parquet/ParquetUtil.java
##########
@@ -136,6 +154,19 @@ public static Metrics footerMetrics(ParquetMetadata
metadata, MetricsConfig metr
toBufferMap(fileSchema, lowerBounds), toBufferMap(fileSchema,
upperBounds));
}
+ private static MessageType getParquetType(ParquetMetadata metadata, Schema
expectedSchema, NameMapping nameMapping) {
+ MessageType type = metadata.getFileMetaData().getSchema();
+ if (ParquetSchemaUtil.hasIds(type)) {
+ return expectedSchema != null ? ParquetSchemaUtil.pruneColumns(type,
expectedSchema) : type;
+ } else if (nameMapping != null) {
+ MessageType typeWithIds = ParquetSchemaUtil.applyNameMapping(type,
nameMapping);
Review comment:
What about a version of `MessageTypeToType` that removes columns without
IDs rather than assigning new ones? That would be more flexible than failing if
unassigned columns are present.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
