korowa commented on code in PR #13223:
URL: https://github.com/apache/datafusion/pull/13223#discussion_r1827020010
##########
datafusion/physical-plan/src/joins/cross_join.rs:
##########
@@ -46,16 +46,23 @@ use
datafusion_physical_expr::equivalence::join_equivalence_properties;
use async_trait::async_trait;
use futures::{ready, Stream, StreamExt, TryStreamExt};
-/// Data of the left side
+/// Data of the left side that is buffered into memory
type JoinLeftData = (RecordBatch, MemoryReservation);
-#[allow(rustdoc::private_intra_doc_links)]
-/// executes partitions in parallel and combines them into a set of
-/// partitions by combining all values from the left with all values on the
right
+/// Cross Join Execution Plan
///
-/// Note that the `Clone` trait is not implemented for this struct due to the
-/// `left_fut` [`OnceAsync`], which is used to coordinate the loading of the
-/// left side with the processing in each output stream.
+/// This operator is used when there are no predicates between two tables and
+/// returns the Cartesian product of the two tables.
+///
+/// Buffers the left input into memory and then streams batches from each
+/// partition on the right input combining them with the buffered left input
+/// to generate the output.
+///
+/// # Clone / Shared State
+///
+/// Note this structure includes a [`OnceAsync`] that is used to coordinate the
+/// loading of the left side with the processing in each output stream.
+/// Therefore it can not be [`Clone`]
Review Comment:
Disregarding this PR -- I wonder is there any difference between cloning
`OnceAsync` and `Arc<dyn ExecutionPlan>` fields?
It just seems that cloning both types is error prone for the cases when
original operator is not dropped after being cloned, and both clone and
original trying to execute -- if so, and there is no significant difference
between these types, maybe we should make `OnceAsync` cloneable, or forbid to
clone almost all `PhysicalPlan`s?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
