JingsongLi commented on code in PR #792:
URL: https://github.com/apache/paimon-rust/pull/792#discussion_r3930300172
##########
crates/paimon/src/table/global_index_scanner/entry.rs:
##########
@@ -169,10 +171,21 @@ pub(super) fn bitmap_meta_may_match(
cmp: &dyn Fn(&[u8], &[u8]) -> Ordering,
) -> bool {
if is_floating_point(data_type) &&
is_bitmap_floating_residual_sensitive_op(op) {
- !meta.only_nulls()
- } else {
- meta.may_match(op, serialized_literals, cmp)
+ return !meta.only_nulls();
}
+ if meta.may_match(op, serialized_literals, cmp) {
+ return true;
+ }
+ // A `±0.0` literal also matches rows holding the other signed zero, but
the
+ // comparator orders `-0.0` below `+0.0`, so a key range that stops at one
of
+ // them would prune a file the reader would have found. `Eq` looks at the
first
+ // literal only, so the alternates are probed as a call of their own rather
+ // than appended.
+ let alternate_zeros: Vec<Vec<u8>> = serialized_literals
Review Comment:
[P1] Apply signed-zero expansion to multivalue metadata pruning too
This expansion only affects `GlobalIndexFileKind::Bitmap`. Array predicates
are routed to `GlobalIndexFileKind::Multivalue`, whose file-level check still
goes through `multivalue_meta_may_match` below using only the original literal.
For a multivalue file whose min/max is `-0.0`, `ARRAY_CONTAINS(+0.0)` (and
similarly `ARRAYS_OVERLAP` / `ARRAY_CONTAINS_ALL`) is therefore rejected before
`BitmapGlobalIndexReader::equal` gets a chance to union the opposite zero. The
scanner then treats the indexed range as covered and returns no candidates, so
matching rows are lost.
I verified this by adding the same `[-0.0, -0.0]` metadata case to
`test_signed_zero_survives_meta_prune` through `multivalue_meta_may_match`; it
fails on `ARRAY_CONTAINS` for `Float`. Please apply the alternate-zero logic
inside the multivalue Eq/In checks as well and cover all three array operators.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
