danny0405 commented on code in PR #19727:
URL: https://github.com/apache/hudi/pull/19727#discussion_r3852123402
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/commit/DatasetBucketRescaleCommitActionExecutor.java:
##########
@@ -57,7 +57,9 @@ public
DatasetBucketRescaleCommitActionExecutor(HoodieWriteConfig config,
*/
@Override
protected BulkInsertPartitioner<Dataset<Row>> getPartitioner(boolean
populateMetaFields, boolean isTablePartitioned) {
- return new
BucketIndexBulkInsertPartitionerWithRows(writeClient.getConfig(), expression,
rule, bucketNumber);
+ return new BucketIndexBulkInsertPartitionerWithRows(
Review Comment:
[P2] Validate custom sort columns on the bucket-rescale path
This override bypasses
`BaseDatasetBulkInsertCommitActionExecutor#getPartitioner`, which is the only
Dataset Row call site for `validateCustomSortColumns`. As a result, an LSM
`BUCKET_RESCALE` write with
`hoodie.bulkinsert.user.defined.partitioner.sort.columns` set is accepted even
though the normal simple/consistent bucket paths reject the same configuration.
The call order makes this more consequential: `execute()` invokes this class’s
`preExecute()` first, and that method persists the new
`PartitionBucketIndexHashingConfig`, before `getPartitioner()` runs. Could this
path perform the same validation before saving the hashing config, with a
rescale regression test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
