taoran92 opened a new pull request, #1458:
URL: https://github.com/apache/paimon-rust/pull/1458

   ### Purpose
   
   Closes #1455.
   
   Let Rust callers fill missing partitions from the branch named by 
`scan.fallback-branch` when batch-reading a partitioned append-only table 
through the filesystem catalog. Reads stay on the current branch by default.
   
   ### Brief change log
   
   - Add `ReadBuilder::with_partition_fallback()` for latest-snapshot batch 
reads. Plan current-branch partition ownership before applying data filters and 
use the backup only for unowned partitions.
   - Route each fallback split through its source branch on the reader. Exclude 
shadowed backup partitions before applying a split limit.
   
   ### Tests
   
   The `partition_fallback_test` integration target passed on the merge with 
`main`. Its overlap case returns ID 1 by default, IDs 1 and 2 with fallback, 
and zero rows for `id = 99`. It also covers transported splits, projection, 
limits, empty branches, and schema changes.
   
   Three additional review cases still fail:
   
   - A planned backup CSV split containing IDs 41 and 42 returns only 42 if the 
source branch enables `csv.include-header` before execution. The reader reopens 
the branch with its latest options and treats the first data row as a header.
   - Setting `scan.snapshot-id` on the source branch after planning makes the 
existing fallback read return `Unsupported`. An ordinary read planned on the 
same source still completes.
   - Changing only the comment on a nested row field makes fallback planning 
reject the two schemas as incompatible.
   
   ### API and Format
   
   Call `with_partition_fallback()` on the builder used for planning and 
reading. Split transport uses Java's existing `FallbackDataSplit` frame (type 
6). Table data and manifest formats are unchanged.
   
   ### Documentation
   
   The new builder method documents the opt-in behavior, supported tables, and 
scope.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to