Akash3121 commented on code in PR #11225:
URL: https://github.com/apache/paimon/pull/11225#discussion_r4603489191


##########
paimon-core/src/main/java/org/apache/paimon/operation/BlobFallbackRecordReader.java:
##########
@@ -120,6 +120,29 @@ public class BlobFallbackRecordReader implements 
RecordReader<InternalRow> {
                     .add(file);
         }
 
+        // Disjoint files cannot supply competing versions of the same row. 
Read them as a

Review Comment:
   The all-disjoint fast path looks good and is safe since real rows keep their 
per-file sequence number and unresolved placeholders still fall through to 
`nullBlobRow(rowId)`.
    
    One thought: because `disjoint` is currently an all-or-nothing check across 
all `files`, if a split has many disjoint base BLOB files (e.g., 100 rolled 
BLOB files `[0, 999]@1`, `[1000, 1999]@2`, ..., `[99000, 99999]@100`) and even 
a **single** sub-range is later updated (e.g., `[0, 999]@101`), `disjoint` 
becomes `false` and the reader falls back to creating 101 sequence-group 
readers instead of 2.
    
    If we instead pack files in descending `maxSequenceNumber` order into the 
earliest group after the last group whose files overlap 
`file.nonNullRowIdRange()`, all-disjoint files still collapse into 1 group, 
while partially updated splits collapse into the minimum number of 
non-overlapping layers (e.g., 2 groups instead of 101) while preserving exact 
descending-sequence precedence for every row ID.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to