deepakpanda93 commented on PR #19471: URL: https://github.com/apache/hudi/pull/19471#issuecomment-5178559396
Thanks @voonhous — the end-to-end verification, especially confirming the holder swap is load-bearing only under `spark.testing`, is the part that would have been easy to mis-measure. And #19487 is a good find. All review points are addressed in `8aac7fd50575`, with replies inline. Summary: | Point | Resolution | | --- | --- | | `allFileSlices2` built with `partitionPath1` | fixed | | Per-partition bucket counts untested | `testCandidateFilesUsePerPartitionBucketCount` added | | Third copy of the holder block | `optimizeResolvedExpr` lifted into the parent, three call sites collapsed | | Duplicated `A = 5 And (A = 2 Or B = 4abc4)` assertion | dropped in both classes | | Partition split random per run | file names ordered before slicing | | `76bb580ae149` context | added to the description | | Fallback case | deferred to #19487, reasoning below | On the fallback: asserting `bucketSet.isEmpty` fails today because of #19487, and asserting the current `Some(Set())` would pin that bug into the suite. It belongs in the fix for #19487 as its regression test rather than here. Verification on Spark 3.5 / Scala 2.12: `TestPartitionBucketIndexSupport` 5/5, `TestBucketIndexSupport` 4/4, scalastyle, checkstyle and rat clean. The control for the new case: pointing the non-matching partition at `2025-06-01`, so both partitions match the expression, fails exactly that test and no other. Your Spark 4.x runs remain the broader coverage; Azure covers the matrix. Also rebased onto master — the branch had fallen behind `d98f2f1b4752`, which removed `HoodieNestedSchemaPruning` that this branch still referenced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
