Akash3121 commented on code in PR #11151:
URL: https://github.com/apache/paimon/pull/11151#discussion_r4579631997


##########
paimon-core/src/test/java/org/apache/paimon/migrate/CopiedDataFilesTest.java:
##########
@@ -323,6 +325,13 @@ public void 
testDedicatedFilesFollowingTheirRowsAreAccepted() throws Exception {
                 .extracting(DataFileMeta::fileName)
                 .containsExactly(
                         "n0.parquet", "v0.vector.parquet", "n1.parquet", 
"v1.vector.parquet");
+        List<ManifestEntry> entries = new ArrayList<>();
+        for (DataFileMeta file : vectors) {
+            entries.add(ManifestEntry.create(FileKind.ADD, 
BinaryRow.EMPTY_ROW, 0, -1, file));
+        }
+        assertThat(RowTrackingCommitUtils.assignRowTracking(1L, 0L, 
entries).assignedEntries)
+                .extracting(entry -> entry.file().firstRowId())
+                .containsExactly(0L, 0L, 100L, 150L);

Review Comment:
   Nice catch preserving `vectorStoreStart` when a copied vector file extends 
past the next normal file's start!
    
    One follow-up question: right below in 
`testDedicatedFilesTheCommitWouldGiveOtherRowsAreRefused` (lines 342–348 and 
362–366), `CopiedDataFiles.checkDedicatedFilesFollowTheirRows` 
(`CopiedDataFiles.java:183-190, 229-236`) still replays the old cursor behavior 
(without `vectorStoreStart = Math.max(vectorStoreStart, start)`) and refuses 
copying tables where a vector column was added after the first normal-only 
files (`// a vector column added after the first rows: the vector-store file 
would get rows 0 and 1`). Should 
`CopiedDataFiles.checkDedicatedFilesFollowTheirRows` also be updated to mirror 
this change (similar to how blob columns added after the first rows are 
accepted in `blobs` below), or do you prefer to keep that for a separate 
follow-up?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to