LuciferYang opened a new issue, #1362: URL: https://github.com/apache/paimon-rust/issues/1362
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Please describe the bug 🐞 For `collect` on an `ARRAY<ROW<...>>` column, Rust compares ROW elements with Arrow `==`, which compares FLOAT and DOUBLE fields bit by bit. Java builds the equaliser with `EqualiserCodeGenerator`, which compares those fields with `Float.compare` / `Double.compare`. The two disagree when a field holds NaN with different payloads: Java treats `ROW(NaN 1)` and `ROW(NaN 2)` as equal, Rust does not. With `distinct=true`, merging `[ROW(NaN 1)]` and `[ROW(NaN 2)]` for one key keeps one element in Java and two in Rust. A retract of `ROW(NaN 2)` removes the stored `ROW(NaN 1)` in Java and nothing in Rust. With `count-limit` set, the extra element can push out a different value. #1352 fixes the same difference for top-level FLOAT/DOUBLE elements. This one has been there since `collect` was added and also applies to nested ROW, ARRAY and MAP elements that contain floating-point fields. ### Solution Compare equaliser-path elements with a function that treats two NaNs in a FLOAT/DOUBLE field as equal and keeps 0.0 and -0.0 apart, matching `Float.compare` / `Double.compare`. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
