danny0405 opened a new pull request, #19992: URL: https://github.com/apache/hudi/pull/19992
### Describe the issue this Pull Request addresses Flink reads column statistics using column-only key prefixes even after partition, bucket, or record-index pruning has narrowed the candidate files. A query selecting one partition can therefore fetch and transpose statistics across the table. ### Summary and Changelog Scope column-stat reads to the partitions containing the remaining file slices. - Pass distinct candidate partition paths from `FileIndex` into the column-stat reader after bucket and record-index pruning. - Build `ColumnStatsIndexPrefixRawKey(column, partition)` prefixes for those partitions. Use a non-null empty list for column-only reads, including partition-stat lookups. - Return before constructing filenames when no column-stat probe exists or no file slices remain. - Preserve conservative inclusion of files missing statistics. - Cover single and multiple partitions, Hive-style and nonpartitioned tables, duplicate partition paths, missing statistics, and the no-probe path in tests. No code was copied. ### Impact Reduces metadata reads and statistics transposition for Flink queries whose candidate files span only part of the table. Filtering results remain unchanged. The change updates Flink-internal statistics method signatures; it introduces no supported user API, configuration, or storage-format changes. ### Risk Level low Incorrect prefix scoping could omit statistics needed for pruning. Tests cover partition naming variants and retain candidates without statistics; existing partition-stat and record-index tests also pass. Validation: - `mvn -o -pl hudi-flink-datasource/hudi-flink -Dtest=TestColumnStatsIndex,TestFileIndex -Dsurefire.rerunFailingTestsCount=0 test`: 49 tests passed. - `mvn -o -pl hudi-flink-datasource/hudi-flink -Dtest=TestColumnStatsIndex -Dsurefire.rerunFailingTestsCount=0 test`: 5 tests passed after the final prefix-builder simplification. - Compilation, Checkstyle, Apache RAT, and `git diff --check` passed. ### Documentation Update none — this is an internal query-planning optimization with no new configuration or user workflow. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
