andygrove opened a new pull request, #7513: URL: https://github.com/apache/datafusion-comet/pull/7513
## Which issue does this PR close? Backport of #6928 to `branch-1.2`. Closes #6799 on this branch. ## Rationale for this change Spark keeps `0.0` and `-0.0` as separate keys in the `percentile` buffer and stable-sorts them as equal, so the zero its hash map iterates first takes the lower ranks. Comet's native percentile picked either zero, so the result could have the wrong sign (#6799, a correctness bug). `branch-1.2` has the same native percentile. See #6928 for the full analysis. ## What changes are included in this PR? Cherry-pick of ca9f52ad53 (`git cherry-pick -x`) with no conflicts and no adaptations. The diff against the source commit is identical apart from line numbers and blob hashes. It adds three fields to the `Percentile` proto message (`float_input`, `signed_zero_keys`, `merge_stage`). The JVM and native sides ship together, so the proto change only needs both halves from this branch. Other release branches: `branch-1.2` is the newest release branch, so there is no newer one to port to. Whether `branch-1.1` needs a backport is not part of this PR. ## How are these changes tested? On this branch, with JDK 17: the 18 `percentile` Rust unit tests pass and `cargo check --workspace --all-targets` is clean (the source PR also edits two benches). The six `percentile` `CometSqlFileTestSuite` files pass against a native library built from this branch. With the six non-test source files reverted to `branch-1.2`'s version, the new `percentile_zero_tie.sql` and `percentile_partials.sql` fail, so the branch has the bug and the tests catch it. I ran the default Spark profile only, and not the Spark 3.5.2 and older profiles that the source PR describes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
