zhoulii commented on code in PR #8945:
URL: https://github.com/apache/paimon/pull/8945#discussion_r3701051812
##########
paimon-common/src/main/java/org/apache/paimon/globalindex/btree/LazyFilteredBTreeReader.java:
##########
@@ -56,6 +56,11 @@ public LazyFilteredBTreeReader(
this.keySerializer = keySerializer;
}
+ @Override
+ public boolean supportsRangeComplement() {
Review Comment:
Do you mean overriding visitIsNotNull, visitNotEqual, and visitNotIn
directly in LazyFilteredBTreeReader?
The challenge is that LazyFilteredBTreeReader only has the index files and
local row IDs, but it does not know the complete row-id range of the shard. A
correct complement requires this full range, which is currently only available
in OffsetGlobalIndexReader as [offset, to].
Implementing the complement directly in LazyFilteredBTreeReader would either
require reading all BTree files to reconstruct the full row-id set, which
defeats this optimization, or passing the row range through
GlobalIndexer#createReader.
That is why the current implementation computes the complement in
OffsetGlobalIndexReader and lets LazyFilteredBTreeReader opt in through
supportsRangeComplement().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
