QuakeWang commented on code in PR #792:
URL: https://github.com/apache/paimon-rust/pull/792#discussion_r3944450195
##########
crates/paimon/src/table/bitmap_global_index_reader.rs:
##########
@@ -270,11 +276,31 @@ impl BitmapGlobalIndexReader {
self.read_bitmap(self.footer.non_null_rows_block).await
}
+ /// Exact dictionary lookup for one key.
async fn equal(&self, key: &[u8], data_type: &DataType) ->
io::Result<RoaringTreemap> {
let logical_cmp = make_bitmap_key_comparator(data_type);
self.equal_with_comparator(key, logical_cmp.as_ref()).await
}
+ /// Scalar equality: the key plus the other signed zero when it is `±0.0`.
+ ///
+ /// Only for the scalar operators. The row-level filter compares a scalar
+ /// through Arrow's IEEE kernel, where `-0.0 == 0.0`, so the index has to
admit
+ /// both keys or it drops rows the filter would keep. Array membership is
+ /// deliberately *bitwise* in `arrow::residual` (Java compares elements
with
+ /// `Float.compareTo`), so the array operators must not widen.
+ async fn equal_including_signed_zero(
+ &self,
+ key: &[u8],
+ data_type: &DataType,
+ ) -> io::Result<RoaringTreemap> {
+ let mut result = self.equal(key, data_type).await?;
+ if let Some(other_zero) = opposite_zero_key(key, data_type) {
+ result |= self.equal(&other_zero, data_type).await?;
Review Comment:
Scalar `Eq`/`In` residuals use Arrow 58.4.0's `eq`, which distinguishes
signed zeros. This widening only adds candidates the residual rejects. Could
you reproduce the claimed false negative with a test that actually runs the
residual filter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
