martin-g commented on code in PR #19668:
URL: https://github.com/apache/datafusion/pull/19668#discussion_r2668107076
##########
datafusion/physical-plan/src/repartition/mod.rs:
##########
@@ -434,33 +434,90 @@ pub const REPARTITION_RANDOM_STATE: SeededRandomState =
SeededRandomState::with_seeds(0, 0, 0, 0);
impl BatchPartitioner {
- /// Create a new [`BatchPartitioner`] with the provided [`Partitioning`]
+ /// Create a new [`BatchPartitioner`] for hash-based repartitioning.
///
- /// The time spent repartitioning will be recorded to `timer`
+ /// # Parameters
+ /// - `exprs`: Expressions used to compute the hash for each input row.
+ /// - `num_partitions`: Total number of output partitions.
+ /// - `timer`: Metric used to record time spent during repartitioning.
+ ///
+ /// # Notes
+ /// This constructor cannot fail and performs no validation.
+ pub fn new_hash_partitioner(
+ exprs: Vec<Arc<dyn PhysicalExpr>>,
+ num_partitions: usize,
+ timer: metrics::Time,
+ ) -> Self {
+ Self {
+ state: BatchPartitionerState::Hash {
+ exprs,
+ num_partitions,
+ hash_buffer: vec![],
+ },
+ timer,
+ }
+ }
+
+ /// Create a new [`BatchPartitioner`] for round-robin repartitioning.
+ ///
+ /// # Parameters
+ /// - `num_partitions`: Total number of output partitions.
+ /// - `timer`: Metric used to record time spent during repartitioning.
+ /// - `input_partition`: Index of the current input partition.
+ /// - `num_input_partitions`: Total number of input partitions.
+ ///
+ /// # Notes
+ /// The starting output partition is derived from the input partition
+ /// to avoid skew when multiple input partitions are used.
+ pub fn new_round_robin_partitioner(
+ num_partitions: usize,
+ timer: metrics::Time,
+ input_partition: usize,
+ num_input_partitions: usize,
Review Comment:
`num_input_partitions == 0` would lead to a panic below (division by 0). It
would be good to prevent this or at least document it as a non-supported value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
