JingsongLi commented on code in PR #8380:
URL: https://github.com/apache/paimon/pull/8380#discussion_r3497381378
##########
paimon-core/src/main/java/org/apache/paimon/deletionvectors/ApplyDeletionFileRecordIterator.java:
##########
@@ -67,7 +69,7 @@ public InternalRow next() throws IOException {
if (next == null) {
return null;
}
- if (!deletionVector.isDeleted(returnedPosition())) {
+ if (!deletionVector.isDeleted(offset + returnedPosition())) {
Review Comment:
The new offset is only applied inside `next()`, but
`DeletionFileRecordIterator` consumers can bypass `next()` and apply
`deletionVector()` themselves. For example,
`ArrowVectorizedBatchConverter.reset(DeletionFileRecordIterator)` iterates the
inner vectorized iterator and checks
`deletionVector.isDeleted(innerIterator.returnedPosition())`, so any
data-evolution reader created with a non-zero offset (for partial
blob/vector-store files or placeholder gaps) will test the wrong DV positions
when exported through that vectorized path. Could we expose an offset-aware
judger here, or otherwise make the offset part of the
`DeletionFileRecordIterator` contract, so all consumers apply the same position
mapping?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
