mbutrovich opened a new pull request, #7690: URL: https://github.com/apache/datafusion-comet/pull/7690
## Which issue does this PR close? Closes #6781. ## Rationale for this change See #6781. ## What changes are included in this PR? `pmod` returns `r + n` for a negative remainder instead of dividing again. `div_round_half_up` computes the remainder from the quotient instead of with a second `i256` division. The PR adds a `pmod` benchmark in the shuffle crate, which calls a new `bench_support::hash_partition_ids`, and a `wide_decimal` multiply case with 48-digit products. ## Benchmark results These are local runs on an Apple M5 Max. They don't predict the Neoverse-N1 and x86-64-v3 builds that Comet releases. Both sides used `-Ctarget-cpu=native` and thin LTO, and they alternated within one run. The aligned layout adds `-C llvm-args=-align-all-functions=7 -C llvm-args=-align-all-nofallthru-blocks=7` to both sides. <!-- Drag ~/Desktop/epic-charts/divide-once-local.png here --> Random hashes are negative half the time, and those rows no longer divide twice. Non-negative hashes never took the second division, so they have nothing to gain. Could a reviewer run the same comparison on x86 with `-Ctarget-cpu=x86-64-v3` and on Neoverse-N1 with `-Ctarget-cpu=neoverse-n1`? On both sides: ```shell cd native RUSTFLAGS="-Ctarget-cpu=x86-64-v3" cargo bench -p datafusion-comet-shuffle --bench pmod RUSTFLAGS="-Ctarget-cpu=x86-64-v3" cargo bench -p datafusion-comet-spark-expr --bench wide_decimal -- '^wide_decimal/fused/multiply' ``` ## How are these changes tested? `test_pmod_edge_cases` in `comet_partitioning.rs` checks `pmod` against `rem_euclid` at the partition boundaries and at `i32::MIN`, `i32::MAX`, and -1. `test_div_round_half_up_matches_reference` in `wide_decimal_binary_expr.rs` checks `div_round_half_up` against the two-division version for divisors from 10 to 10^76, at exact halves and at the `i128` and `i256` bounds. Both tests pass on `main` too. # AI usage disclosure I wrote this PR with help from Claude but understand and support the core approach, implementation, and test coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
