voonhous opened a new issue, #20633:
URL: https://github.com/apache/hudi/issues/20633

   Part of #20625. Line references are against master `db90a0da`.
   
   ### Problem
   The merge path reads the base file with `enablePredicatePushDown=false` 
(`HudiPageSourceProvider.java:258`), so the Parquet tuple domain is 
`TupleDomain.all()`: static predicates and dynamic filters prune no row groups 
or pages. Partition-level dynamic filtering in `HudiSplitSource` still applies 
to MOR splits; the loss is pruning inside the file.
   
   ### Direction
   Enable base-file pushdown only where it cannot change the merged result, 
with Trino re-applying the filter after the merge:
   
   - record-key-only filters (what Spark and Flink push under MOR);
   - any filter under `COMMIT_TIME_ORDERING` with full-row log records.
   
   Keep it off for `EVENT_TIME_ORDERING`, partial updates and custom mergers, 
where a pruned base row can let a log record surface as a spurious row.
   
   ### Exit criteria
   - Pushdown enabled only for the safe cases; the eligibility decision is 
unit-tested.
   - Correctness tests where an update moves a row into and out of the 
predicate, and where a delete targets a row whose base row group would be 
pruned; merged results equal a no-pushdown read.
   - Benchmark evidence on multi-row-group base files (generator option) that 
physical input bytes drop for a selective predicate on scan `_rt`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to