peterxcli opened a new pull request, #24417: URL: https://github.com/apache/datafusion/pull/24417
## Which issue does this PR close? per https://github.com/apache/datafusion/pull/23854#discussion_r3758046872 > as a follow on it might be nice to break this function into smaller functions (mostly so they can be documented more clearly) -- the techniques here are quite clever ## Rationale for this change 1. Code in `SharedBuildAccumulator.build_filter` is too complicated and too clever to understand, need to extract it into smaller pieces of logic block. 2. While trying to refactor the code, I realized my previous pr https://github.com/apache/datafusion/pull/23854 didn't follow the sparse routing branches build that hash partitioning is using, because I think I didnt notice that's a thing atm. so I decided to deliever the fix for range partition along with the refactor, because then both range and hash partition dynamic filter expression would have almost similar building structure. ## What changes are included in this PR? - Apply same sparse routing for range partition dynamic filter that hash partition, so we can share more code among them, now there difference is literally only at the top physical expression to decide which partition should the `key` go. - Extract `build_filter` in `shared_bounds` to `build_collect_left_filter` and `build_partitioned_filter`, and add comment explanation throughout the code. The most complicated part are filter building fast-path(`all_partitions_canceled`, `all_partitions_empty` and `one_non_empty_partition`) and sparse branches filter building. btw, I skipped documenting the state management for `PartitionStatus` because I think its code is already self-explanatory. ## Are these changes tested? - test mod share_bounds - filter_pushdown test - range_partition.slt ## Are there any user-facing changes? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
