rdblue commented on a change in pull request #2284:
URL: https://github.com/apache/iceberg/pull/2284#discussion_r603678133
##########
File path: core/src/main/java/org/apache/iceberg/TableMetadata.java
##########
@@ -842,18 +844,42 @@ private static SortOrder updateSortOrderSchema(Schema
schema, SortOrder sortOrde
return builder.build();
}
- private static PartitionSpec freshSpec(int specId, Schema schema,
PartitionSpec partitionSpec) {
+ private static PartitionSpec freshSpec(int specId, Schema schema,
PartitionSpec partitionSpec, int formatVersion,
+ List<PartitionSpec> specs,
AtomicInteger lastPartitionId) {
PartitionSpec.Builder specBuilder = PartitionSpec.builderFor(schema)
.withSpecId(specId);
- for (PartitionField field : partitionSpec.fields()) {
- // look up the name of the source field in the old schema to get the new
schema's id
- String sourceName =
partitionSpec.schema().findColumnName(field.sourceId());
- specBuilder.add(
- schema.findField(sourceName).fieldId(),
- field.fieldId(),
- field.name(),
- field.transform().toString());
+ if (formatVersion > 1) {
+ Map<Pair<Integer, String>, Integer> transformToFieldId = specs.stream()
+ .flatMap(spec -> spec.fields().stream())
+ .collect(Collectors.toMap(
+ field -> Pair.of(field.sourceId(), field.transform().toString()),
+ PartitionField::fieldId,
+ (n1, n2) -> n2));
+
+ for (PartitionField field : partitionSpec.fields()) {
Review comment:
This logic looks mostly correct for a `reassignPartitionIds` method, but
I would separate the concern of updating the source field ID references (what
`freshSpec` originally did) from reassigning partition IDs to be consistent
with other specs in the table. We should use `freshSpec` to reassign the source
IDs to the ones used in the current table schema, and then we should call
`reassignPartitionIds` to make them match existing partition specs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
