yihua opened a new pull request, #984:
URL: https://github.com/apache/hudi-rs/pull/984

   ## Description
   
   An incremental read took each touched file group's slice from a snapshot at 
the last admitted commit's requested time, built over every completed commit. A 
commit requested before that point but completed after the window (OCC 
multi-writer, or a late table service) could then supply the slice, the 
commit-time mask dropped its rows, and the window's version of the row was 
gone. `resolve_incremental_window` now also carries the window end as given, 
and incremental planning (touched file groups, replaced file groups and file 
visibility) runs on a timeline truncated to commits completed by that bound via 
`Timeline::as_of_completion_time`. It is a no-op on layout v1, and snapshot and 
time-travel planning are unchanged. The incremental note in AGENTS.md is 
updated to match.
   
   For regular commits this matches Hudi 1.x, whose incremental file system 
view only holds files written by the admitted commits. It deliberately differs 
for a replacecommit requested inside the window but completed after it: Java's 
replaced-file-group set comes from the full timeline, so its answer flips once 
that commit completes, while this returns the same rows either way.
   
   Closes #922
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   A case-table test uses native writes with out-of-order completion (late 
upsert on COW and MOR, late partition overwrite, late compaction) and asserts 
each window returns the same rows, and `get_file_slices` plans the same slices, 
before and after the late commit completes; on main the upsert and overwrite 
cases lose the row. Unit tests cover `Timeline::as_of_completion_time` on both 
layouts and that the window end comes only from resolution.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to