jianguotian opened a new pull request, #850: URL: https://github.com/apache/paimon-rust/pull/850
## Purpose Bucket predicates are currently evaluated only after each manifest file is fetched and decoded. For selective fixed-bucket queries, especially single-key lookups over bucket-sorted manifests, this leaves substantial planning I/O on the critical path. ## Changes - Add the Java-compatible nullable `_TOTAL_BUCKETS` field to `ManifestFileMeta`. - Persist it only when every manifest entry has the same positive bucket count. - Before fetching manifest contents, combine `_MIN_BUCKET`, `_MAX_BUCKET`, and `_TOTAL_BUCKETS` with the existing bucket-key predicate hash calculation. - Apply the same pruning to full snapshot and incremental manifest-list scans. - Fail open for legacy, missing, invalid, mixed, or unassigned-bucket metadata. - Retain existing entry-level filtering as a correctness backstop. This complements bucket-first manifest sorting: sorting tightens each manifest's bucket envelope, while this PR lets the Rust planner exploit that envelope. `_TOTAL_BUCKETS` follows Java PR apache/paimon#9791. PR #848 adds the adjacent `_EXTRA_FILES` field independently; this branch intentionally keeps the two review units separate and will be rebased for canonical final schema order after either one lands. ## Verification - `cargo test -p paimon --lib`: 2,818 passed, 2 ignored - `cargo clippy -p paimon --lib --tests -- -D warnings` - Regression coverage for exact bucket selection, legacy fail-open behavior, unassigned buckets, and mixed/invalid total bucket counts ## Related Java work - apache/paimon#9791 - apache/paimon#9792 - apache/paimon#9802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
