JTaky opened a new pull request, #8335:
URL: https://github.com/apache/paimon/pull/8335
Option-only alterTable calls (e.g. changing metadata.iceberg.* properties)
increment Paimon's schema ID without modifying columns, producing multiple
IcebergSchema objects with identical fields but different IDs.
Root cause: TableMetadata.Builder.addSchema() uses sameSchema() to
deduplicate identical schemas internally, collapsing them to the first ID. When
setCurrentSchema(N) is called with the higher (non-surviving) ID, it fails:
"Cannot set current schema to unknown schema: N"
Fix summary:
- Deduplicate schemas in adjustMetadataForRest() before passing to Iceberg.
Converts to Iceberg Schema objects and uses Schema.sameSchema() directly, so
dedup semantics always match Iceberg's internal reuseOrCreateNewSchemaId() —
including any future changes to sameSchema().
- Renumber surviving schemas sequentially (1, 2, 3, ...) to match Iceberg's
sequential ID assignment and avoid setCurrentSchema() failures due to gaps.
- Remap currentSchemaId and all snapshot schemaId references through the
combined dedup + renumber map.
- Fix NPE in commitMetadataImpl: when the Iceberg table exists but has no
snapshots (e.g. after a failed commit post-recreateTable), call
updatesForCorrectBase(isNewTable=true) directly instead of routing through
updatesForIncorrectBase → recreateTable, which would cause an infinite
drop+create loop on repeated commit failures.
- Update Iceberg REST table properties only when they differ from the
current catalog state, avoiding redundant SetProperties updates on every commit.
Fixes: https://github.com/apache/paimon/issues/6906
### Purpose
### Tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
