yihua opened a new issue, #909: URL: https://github.com/apache/hudi-rs/issues/909
### Feature Description A table read with filters decodes every column of every file slice and applies the filters as a row mask afterwards. The parquet reader can evaluate a predicate on the filter columns first and decode the rest only for the rows that pass, and it can skip row groups whose statistics rule the filter out; the merge-on-read reader already carries the hooks for both, but nothing sets them from `ReadOptions.filters`. A key lookup on a table of wide rows therefore costs a full decode of each file it opens. Proposed: compile the caller's filters into the reader's row filter and row-group selector for every slice read, pushing only what cannot change the result (a file storing the filter columns in the table's types; only key filters when log files have to merge; no range pruning on float statistics), and keep applying the filters after the merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
