rangareddy commented on issue #17392: URL: https://github.com/apache/hudi/issues/17392#issuecomment-5351243890
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9129). **Findings: partially delivered - key-based `IN` filtering works, general expression filters do not.** A predicate is now pushed into the metadata table read path. `hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:711` builds one: ```java Predicate predicate = buildPredicate(partitionName, sortedKeys, isFullKey); ``` and `buildPredicate` at `:736` returns `Predicates.in(...)` (`:742`), which is passed into the `HoodieAvroReaderContext` as `Option.of(predicate)` alongside the base file readers. But the supported shape is narrow, and the code says so explicitly at `:603`: ```java ValidationUtils.checkArgument(predicate instanceof Predicates.In, "For Metadata Table Reuse, key filter should be based on full keys"); ``` So what exists is key lookup expressed as a predicate, not expression filtering. Range predicates, conjunctions, and any filter over non-key fields are not supported, and the reuse path actively rejects anything that is not `Predicates.In`. This ticket is therefore still live, with the scope now clearer than when it was filed: extend the MDT reader context beyond full-key `IN` predicates, and relax that `checkArgument` accordingly. Sibling context - #17393 (HUDI-9130) is done, #17391 (HUDI-9128) was delivered under HUDI-9592, and #17394 (HUDI-9131) is the benchmark that depends on this ticket landing first. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
