Dandandan commented on a change in pull request #9605:
URL: https://github.com/apache/arrow/pull/9605#discussion_r584888850
##########
File path: rust/datafusion/src/physical_plan/repartition.rs
##########
@@ -48,9 +51,13 @@ pub struct RepartitionExec {
input: Arc<dyn ExecutionPlan>,
/// Partitioning scheme to use
partitioning: Partitioning,
- /// Channels for sending batches from input partitions to output partitions
- /// there is one entry in this Vec for each output partition
- channels: Arc<Mutex<Vec<(Sender<MaybeBatch>, Receiver<MaybeBatch>)>>>,
+ /// Channels for sending batches from input partitions to output
partitions.
+ /// Key is the partition number
+ channels: Arc<
Review comment:
I didn't follow the answer completely; why do we need to remove it?
I think `UnboundedSender` is cheap to clone and cheap to keep in memory for
the duration of `RepartitionStream`?
But maybe I would need to play with the code more to see why it is needed
per se.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
