YannByron opened a new pull request, #1403:
URL: https://github.com/apache/paimon-rust/pull/1403

   ### Purpose
   
   Closes #1402. Follow-up to #901.
   
   Data Evolution reads already preselect row IDs before materializing managed 
BLOBs for ordinary-column predicates with a `LIMIT`. Without a limit, they 
bypass that path and decode payloads that the residual predicate discards. 
Extend the same preselection to unbounded filtered reads, using current column 
values.
   
   ### Brief change log
   
   - Remove the limit-only eligibility check and keep the remaining row quota 
optional across splits.
   - Preserve the existing exclusions for descriptor reads, BLOB-dependent 
predicates, BLOB views and legacy files without row IDs.
   - Add a regression with updated scalar values, one or multiple BLOB files, 
small batches and zero matches. An excluded payload has an invalid entry 
header, so the test fails if that payload is decoded.
   
   The two-pass path trades additional ordinary-column work for avoiding 
unselected payload materialization; this PR does not claim a speedup for every 
selectivity.
   
   ### Tests
   
   Against main at `8a432a6398ce9e19a57e595b79dcde4d64304bc4`, the new 
regression fails before the production change with `Invalid Blob entry magic` 
for the excluded payload. With this change:
   
   - `cargo +1.95.0 test --locked -p paimon --lib 
table::data_evolution_reader::`: **87 passed**.
   - `cargo +1.95.0 test --locked -p paimon --lib blob`: **264 passed**. These 
filters overlap.
   - `cargo +1.95.0 fmt --all -- --check`: passed.
   - `git diff --check`: passed.
   
   The tests use filesystem-backed fixtures and need no cloud credentials.
   
   ### API and Format
   
   No public API or storage-format changes. Existing limited reads retain their 
quota behavior.
   
   ### Documentation
   
   No new options or user-facing API; no documentation changes required.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to