aokolnychyi commented on a change in pull request #1335:
URL: https://github.com/apache/iceberg/pull/1335#discussion_r470037828
##########
File path: parquet/src/main/java/org/apache/iceberg/parquet/ParquetUtil.java
##########
@@ -136,6 +154,19 @@ public static Metrics footerMetrics(ParquetMetadata
metadata, MetricsConfig metr
toBufferMap(fileSchema, lowerBounds), toBufferMap(fileSchema,
upperBounds));
}
+ private static MessageType getParquetType(ParquetMetadata metadata, Schema
expectedSchema, NameMapping nameMapping) {
+ MessageType type = metadata.getFileMetaData().getSchema();
+ if (ParquetSchemaUtil.hasIds(type)) {
+ return expectedSchema != null ? ParquetSchemaUtil.pruneColumns(type,
expectedSchema) : type;
+ } else if (nameMapping != null) {
+ MessageType typeWithIds = ParquetSchemaUtil.applyNameMapping(type,
nameMapping);
Review comment:
We have to prune columns without ids. Otherwise,
`ParquetSchemaUtil.convert` will assign fallback ids to columns not present in
the name mapping and they will most likely conflict with ids assigned by the
name mapping.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
