JingsongLi commented on code in PR #7865: URL: https://github.com/apache/paimon/pull/7865#discussion_r3458901133
########## docs/docs/maintenance/rescale-bucket.md: ########## @@ -45,14 +45,17 @@ Please note that - `ALTER TABLE` only modifies the table's metadata and will **NOT** reorganize or reformat existing data. Reorganize existing data must be achieved by `INSERT OVERWRITE`. - Rescale bucket number does not influence the read and running write jobs. -- Once the bucket number is changed, any newly scheduled `INSERT INTO` jobs which write to without-reorganized - existing table/partition will throw a `TableException` with message like +- **Partitioned tables** support per-partition bucket counts. Each partition retains its own bucket Review Comment: This statement is broader than the implementation today. The Flink fixed-bucket path now routes through `table.createRowKeyExtractor()`, but the Spark extension fast path still computes `BUCKET_COL` with the table-level `coreOptions.bucket()` in `PaimonSparkWriter` (`HASH_FIXED`, `paimonExtensionEnabled`). After changing the table default from 4 to 8 while an existing partition still has 4 buckets, Spark can route keys for that old partition to buckets 4-7, and the write-side mismatch check is now relaxed for partitioned tables. That breaks the promised per-partition layout and can create files in buckets that do not belong to the partition. Please either update the Spark fixed-bucket path to use the same per-partition bucket mapping, or scope the docs/procedure support to engines that actually use it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
