jackylee-ch opened a new pull request, #1422:
URL: https://github.com/apache/paimon-rust/pull/1422

   ### Purpose
   
   Linked issue: N/A (bug fix)
   
   Reading the window of `incremental-between-timestamp` of a data-evolution 
table fails when a compaction merged the files of two inserts and an update 
then covered the merged rows: "All data files in a field merge split should 
have the same row id range". The window holds the two inserts' files and the 
update's file, which are not column slices of one row range, and the reader 
tried to merge them into one group.
   
   ### Brief change log
   
   - `DataEvolutionReader` reads the files of a streaming (incremental) group 
that is not a column slice of one row range as they are stored, as it does for 
raw-convertible groups.
   - New `is_data_evolution_column_slice_group` decides this: every normal file 
covers the same rows and every BLOB or vector file lies inside them. Real 
column slices (an insert and an update of its rows) still merge, and ordinary 
reads are unchanged.
   
   ### Tests
   
   - Added `data_evolution_window_test.rs`: a window of an insert and a column 
update of its rows merges into the updated rows; a window of two inserts, a 
compaction and an update of every row reads the eight stored rows instead of 
failing. The second test fails on `main` and passes with this change.
   - Added a unit test for `is_data_evolution_column_slice_group`.
   - `cargo test -p paimon` passes; `cargo fmt --check` is clean.
   
   ### API and Format
   
   No API or storage format change.
   
   ### Documentation
   
   No documentation change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to