LuciferYang commented on code in PR #10643:
URL: https://github.com/apache/paimon/pull/10643#discussion_r4314968508
##########
paimon-core/src/main/java/org/apache/paimon/operation/DataEvolutionSplitRead.java:
##########
@@ -778,6 +808,31 @@ private static int findBlobFieldIndex(RowType rowType) {
return -1;
}
+ /**
+ * Emits the rows of blob files for a read type with no blob field,
without opening them: the
+ * blob format cannot be read without one.
+ */
+ private FileRecordReader<InternalRow> blobRowsWithoutBlobField(
+ Range range,
+ List<DataFileMeta> files,
+ List<Range> rowRanges,
+ RowType readRowType,
+ @Nullable DeletionVectorWithRange deletionVector) {
+ long sequenceNumber =
+
files.stream().mapToLong(DataFileMeta::maxSequenceNumber).max().getAsLong();
Review Comment:
Fixed in 2438de2d0a. When `_SEQUENCE_NUMBER` is read without the blob field,
the blob files now go through the same `BlobFallbackRecordReader` with the blob
read as a descriptor, and the blob field is dropped afterwards, so each row
gets the version of the newest file holding a value for it. File ranges alone
are not enough here: a Flink partial update writes placeholders for the rows it
did not change. A single blob file is now merged the same way, so its
placeholders are resolved too.
Your repro is `testIncrementalReadOfBlobOnlyDeltasKeepsRowVersions`, which
compares the tracking fields with and without the BLOB projection.
`testIncrementalReadOfBlobOnlyDeltaWithPlaceholders` covers a blob file that
holds placeholders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
