Dandandan opened a new pull request, #21900: URL: https://github.com/apache/datafusion/pull/21900
## Which issue does this PR close? - Closes #. ## Rationale for this change Hash repartition assigns each row with . Because is a runtime value, this can compile to hardware division in the per-row partitioning loop. This is an alternative to #21830 that preserves the existing modulo partition mapping by strength-reducing the u64 remainder instead of changing the mapping to fastrange. ## What changes are included in this PR? - Adds a small local u64 strength reducer for exact remainder computation. - Uses a mask for power-of-two partition counts, including 1. - Uses a precomputed reciprocal multiply path for other partition counts. - Validates hash partitioner construction rejects zero partitions. - Uses the precomputed reducer in 's hash index distribution loop. ## Are these changes tested? - running 1 test test repartition::tests::strength_reduced_u64_remainder_matches_modulo ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1391 filtered out; finished in 0.01s - running 1 test test repartition::tests::hash_partitioner_requires_nonzero_partitions ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1391 filtered out; finished in 0.00s - running 1 test test repartition::tests::many_to_many_hash_partition ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1391 filtered out; finished in 0.01s - running 21 tests test repartition::tests::hash_partitioner_requires_nonzero_partitions ... ok test repartition::tests::error_for_input_exec ... ok test repartition::tests::oom ... ok test repartition::tests::repartition_with_error_in_stream ... ok test repartition::tests::hash_repartition_avoid_empty_batch ... ok test repartition::tests::one_to_many_round_robin ... ok test repartition::tests::many_to_many_round_robin ... ok test repartition::tests::many_to_many_round_robin_within_tokio_task ... ok test repartition::tests::many_to_one_round_robin ... ok test repartition::tests::repartition_without_spilling ... ok test repartition::tests::hash_repartition_with_dropping_output_stream ... ok test repartition::tests::unsupported_partitioning ... ok test repartition::tests::test_repartition_with_coalescing ... ok test repartition::tests::many_to_many_hash_partition ... ok test repartition::tests::strength_reduced_u64_remainder_matches_modulo ... ok test repartition::tests::test_repartition_ordering_with_spilling ... ok test repartition::tests::repartition_with_partial_spilling ... ok test repartition::tests::repartition_with_spilling ... ok test repartition::tests::test_drop_cancel ... ok test repartition::tests::robin_repartition_with_dropping_output_stream ... ok test repartition::tests::repartition_with_delayed_stream ... ok test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 1371 filtered out; finished in 0.14s - - - [rust_lint.sh] Running rust_fmt.sh [rust_fmt.sh] `cargo fmt --all -- --check` [rust_lint.sh] Running rust_clippy.sh [rust_clippy.sh] `cargo clippy --all-targets --workspace --features avro,integration-tests,extended_tests -- -D warnings` [rust_lint.sh] Running rust_toml_fmt.sh [rust_toml_fmt.sh] `taplo format --check` [rust_lint.sh] Running license_header.sh [license_header.sh] `hawkeye check --config licenserc.toml` [rust_lint.sh] Running typos_check.sh [typos_check.sh] `typos --config typos.toml` [rust_lint.sh] Running doc_prettier_check.sh [doc_prettier_check.sh] [email protected] check Checking formatting... All matched files use Prettier code style! [rust_lint.sh] Running rust_docs.sh ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
