LuciferYang opened a new issue, #1362:
URL: https://github.com/apache/paimon-rust/issues/1362

   ### Search before asking
   
   - [x] I searched in the 
[issues](https://github.com/apache/paimon-rust/issues) and found nothing 
similar.
   
   ### Please describe the bug 🐞
   
   For `collect` on an `ARRAY<ROW<...>>` column, Rust compares ROW elements 
with Arrow `==`, which compares FLOAT and DOUBLE fields bit by bit. Java builds 
the equaliser with `EqualiserCodeGenerator`, which compares those fields with 
`Float.compare` / `Double.compare`. The two disagree when a field holds NaN 
with different payloads: Java treats `ROW(NaN 1)` and `ROW(NaN 2)` as equal, 
Rust does not.
   
   With `distinct=true`, merging `[ROW(NaN 1)]` and `[ROW(NaN 2)]` for one key 
keeps one element in Java and two in Rust. A retract of `ROW(NaN 2)` removes 
the stored `ROW(NaN 1)` in Java and nothing in Rust. With `count-limit` set, 
the extra element can push out a different value.
   
   #1352 fixes the same difference for top-level FLOAT/DOUBLE elements. This 
one has been there since `collect` was added and also applies to nested ROW, 
ARRAY and MAP elements that contain floating-point fields.
   
   ### Solution
   
   Compare equaliser-path elements with a function that treats two NaNs in a 
FLOAT/DOUBLE field as equal and keeps 0.0 and -0.0 apart, matching 
`Float.compare` / `Double.compare`.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to