zhang-arvin opened a new pull request, #10077: URL: https://github.com/apache/paimon/pull/10077
### What is the purpose of the change Fixes #9993. The automatic schema merge path bypasses the invariant that the explicit schema change path enforces: key columns must not change type. ### Brief change log - `SchemaMergingUtils#mergeSchemas(TableSchema, RowType, ...)` merges every incoming field type and then re-attaches the **existing** `primaryKeys()` / `partitionKeys()` name lists, without ever validating that those columns kept their type. `SchemaManagerUtils` already rejects this for explicit changes via `assertNotUpdatingPartitionKeys` / `assertNotUpdatingPrimaryKeys`, but `FileSystemSchemaManager#mergeSchema` (i.e. every write-time automatic schema merge) called `mergeSchemas` directly and skipped those guards. - Add `assertNotUpdatingKeyColumn`, invoked right before the merged `TableSchema` is constructed, which rejects the merge with an `UnsupportedOperationException` when a primary-key or partition column's type changed. - Non-key column widening behaviour is unchanged. ### Why it matters Widening a key column changes its value encoding, so the same logical key hashes into a different bucket. Concretely, for a bucketed primary-key table, `INT(-100000)` hashes to bucket 3 while `BIGINT(-100000)` hashes to bucket 13 — the same logical key ends up in two different buckets, so intra-bucket deduplication no longer applies and duplicate rows accumulate. ### Tests - `testRejectTypeWideningOnPrimaryKeyColumn` — covers the reported `INT -> BIGINT` primary-key scenario, asserting the merge is rejected. - `testRejectTypeWideningOnPartitionColumn` — same for a partition column. - `testAllowTypeWideningOnNonKeyColumn` — guards against over-rejection: non-key widening still merges. All 17 tests in `SchemaMergingUtilsTest` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
