JingsongLi commented on code in PR #7865:
URL: https://github.com/apache/paimon/pull/7865#discussion_r3458905923
##########
paimon-core/src/main/java/org/apache/paimon/operation/AbstractFileStoreWrite.java:
##########
@@ -552,20 +552,28 @@ private RestoreFiles scanExistingFileMetas(BinaryRow
partition, int bucket) {
totalBuckets = restoredTotalBuckets;
}
if (!ignoreNumBucketCheck && totalBuckets != numBuckets) {
- String partInfo =
- partitionType.getFieldCount() > 0
- ? "partition "
- + getPartitionComputer(
- partitionType,
-
PARTITION_DEFAULT_NAME.defaultValue(),
- legacyPartitionName)
- .generatePartValues(partition)
- : "table";
- throw new RuntimeException(
- String.format(
- "Try to write %s with a new bucket num %d, but the
previous bucket num is %d. "
- + "Please switch to batch mode, and
perform INSERT OVERWRITE to rescale current data layout first.",
- partInfo, numBuckets, totalBuckets));
+ if (partitionType.getFieldCount() > 0) {
Review Comment:
This relaxation still needs to reject buckets outside the partition layout.
When `restoreFiles()` returns no entries for an empty bucket,
`WriteRestore.extractTotalBuckets` can now recover the partition bucket count
from `PartitionBucketMapping`. For an old partition with 4 buckets after the
table default was changed to 8, a caller that still computes buckets with the
schema default can ask to write bucket 6. This branch only logs the 4-vs-8
mismatch and then continues, so the writer can create files in bucket 6 even
though that partition should only have buckets 0-3. Please keep the
per-partition mismatch allowance, but fail when `bucket >= totalBuckets`
(unless `ignoreNumBucketCheck` is explicitly set for the controlled
rescale/postpone path).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
