YannByron opened a new pull request, #1403: URL: https://github.com/apache/paimon-rust/pull/1403
### Purpose Closes #1402. Follow-up to #901. Data Evolution reads already preselect row IDs before materializing managed BLOBs for ordinary-column predicates with a `LIMIT`. Without a limit, they bypass that path and decode payloads that the residual predicate discards. Extend the same preselection to unbounded filtered reads, using current column values. ### Brief change log - Remove the limit-only eligibility check and keep the remaining row quota optional across splits. - Preserve the existing exclusions for descriptor reads, BLOB-dependent predicates, BLOB views and legacy files without row IDs. - Add a regression with updated scalar values, one or multiple BLOB files, small batches and zero matches. An excluded payload has an invalid entry header, so the test fails if that payload is decoded. The two-pass path trades additional ordinary-column work for avoiding unselected payload materialization; this PR does not claim a speedup for every selectivity. ### Tests Against main at `8a432a6398ce9e19a57e595b79dcde4d64304bc4`, the new regression fails before the production change with `Invalid Blob entry magic` for the excluded payload. With this change: - `cargo +1.95.0 test --locked -p paimon --lib table::data_evolution_reader::`: **87 passed**. - `cargo +1.95.0 test --locked -p paimon --lib blob`: **264 passed**. These filters overlap. - `cargo +1.95.0 fmt --all -- --check`: passed. - `git diff --check`: passed. The tests use filesystem-backed fixtures and need no cloud credentials. ### API and Format No public API or storage-format changes. Existing limited reads retain their quota behavior. ### Documentation No new options or user-facing API; no documentation changes required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
