fhan688 commented on code in PR #3501:
URL: https://github.com/apache/fluss/pull/3501#discussion_r3446593547
##########
fluss-lake/fluss-lake-hudi/src/main/java/org/apache/fluss/lake/hudi/source/HudiLakeSource.java:
##########
Review Comment:
> do we need to pass `predicates` to planner to reduce the plan splits?
Good point. Passing predicates to the planner would help reduce planned
splits, similar to Iceberg/Paimon.
For this PR, the predicates are applied in `HudiRecordReader`, so the
result correctness is preserved, but split planning still scans all
partition/file slices. Hudi's current planner path uses
`HoodieTableFileSystemView` directly, and the converted
`ExpressionPredicates` are mainly reader/parquet predicates rather than a
planner-level scan filter API.
I think planner-level pruning, especially partition pruning, is a valid
follow-up optimization. If you prefer, I can keep this PR focused on
reader-level predicate pushdown and leave Hudi split pruning to a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
