aokolnychyi commented on a change in pull request #3411:
URL: https://github.com/apache/iceberg/pull/3411#discussion_r821254517
##########
File path: core/src/main/java/org/apache/iceberg/Partitioning.java
##########
@@ -211,6 +211,8 @@ public static StructType partitionType(Table table) {
Map<Integer, PartitionField> fieldMap = Maps.newHashMap();
List<NestedField> structFields = Lists.newArrayList();
+ // Mapping from a PartitionField with void transform to the index in
structFields.
Review comment:
It is a little bit hard to follow these changes and they also break the
logic for inheriting updated column names.
What about something like this?
```
Map<Integer, PartitionField> fieldMap = Maps.newHashMap();
Map<Integer, Type> typeMap = Maps.newHashMap();
Map<Integer, String> nameMap = Maps.newHashMap();
// sort the spec IDs in descending order to pick up the most recent field
names
List<Integer> specIds = table.specs().keySet().stream()
.sorted(Collections.reverseOrder())
.collect(Collectors.toList());
for (Integer specId : specIds) {
PartitionSpec spec = table.specs().get(specId);
for (PartitionField field : spec.fields()) {
int fieldId = field.fieldId();
NestedField structField = spec.partitionType().field(fieldId);
PartitionField existingField = fieldMap.get(fieldId);
if (existingField == null) {
fieldMap.put(fieldId, field);
typeMap.put(fieldId, structField.type());
nameMap.put(fieldId, structField.name());
} else {
// verify the fields are compatible as they may conflict in v1 tables
ValidationException.check(equivalentIgnoringNames(field,
existingField),
"Conflicting partition fields: ['%s', '%s']",
field, existingField);
// use the correct type for dropped partitions in v1 tables
if (isVoidTransform(existingField) && !isVoidTransform(field)) {
fieldMap.put(fieldId, field);
typeMap.put(fieldId, structField.type());
}
}
}
}
List<NestedField> sortedStructFields = fieldMap.keySet().stream()
.sorted(Comparator.naturalOrder())
.map(fieldId -> NestedField.optional(fieldId, nameMap.get(fieldId),
typeMap.get(fieldId)))
.collect(Collectors.toList());
return StructType.of(sortedStructFields);
```
```
private static boolean isVoidTransform(PartitionField field) {
return field.transform().equals(Transforms.alwaysNull());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
