plusplusjiajia opened a new pull request, #10871: URL: https://github.com/apache/paimon/pull/10871
### Purpose When writing to a partition, PyPaimon restored the sequence numbers of its existing files but did not check the bucket count those files were written with. After the `bucket` option was changed, for example from 2 to 4, writes to an existing partition used the new bucket count. Primary keys already stored under the old layout were written again to other buckets, and the table returned duplicate keys. As in Java `AbstractFileStoreWrite`, writing to a bucket whose existing files have a different total bucket number now fails with Java's message, which asks to rescale with `INSERT OVERWRITE` first. The check reuses the existing per-partition restore scan. Overwrite does not restore existing files, like Java's `ignorePreviousFiles`, so it is not affected. ### Tests - `TableWriteTest#test_write_bucket_count`: writing to a non-partitioned and a partitioned table after changing the bucket count fails with Java's message, and overwrite still rescales the data. - `TableWriteTest#test_postpone_checks_restored_bucket_count`: the postpone fixed-bucket writer checks the bucket count of restored files and skips the check for overwrite. - `trusted_scan_metadata_test.py`: the restore scan test sets the new writer fields. - 3 of the new test cases fail on master; the overwrite case guards existing behavior. The related tests pass locally: 80 in `table_write_test.py` and `trusted_scan_metadata_test.py`, and 1276 in the write and bucket tests. flake8 is clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
