rdblue commented on a change in pull request #3421:
URL: https://github.com/apache/iceberg/pull/3421#discussion_r739884197
##########
File path: core/src/main/java/org/apache/iceberg/TableMetadata.java
##########
@@ -743,19 +743,23 @@ private PartitionSpec reassignPartitionIds(PartitionSpec
partitionSpec, TypeUtil
} else {
// for v1, preserve the existing spec and carry forward all fields,
replacing missing fields with void
- Map<Pair<Integer, String>, PartitionField> newFields =
Maps.newLinkedHashMap();
+ Map<Integer, PartitionField> newFields = Maps.newLinkedHashMap();
for (PartitionField newField : partitionSpec.fields()) {
- newFields.put(Pair.of(newField.sourceId(),
newField.transform().toString()), newField);
+ newFields.put(newField.sourceId(), newField);
}
for (PartitionField field : spec().fields()) {
// ensure each field is either carried forward or replaced with void
- PartitionField newField = newFields.remove(Pair.of(field.sourceId(),
field.transform().toString()));
Review comment:
Why did you remove the transform string from the dedup? The fields are
different if they have different transforms so I don't think that this is
correct. For example, if the current spec is `[ 1000: bucket[16](2: uuid) ]`
and the incoming spec is `[ 1000: identity(2: uuid) ]` then the result must be
`[ 1000: always_null(2: uuid), 1001: identity(2: uuid) ]`. That's because the
output of `identity(uuid)` is a string and the output of `bucket[16](uuid)` is
an int. If we tried to read the files table after this change, it would fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
