alamb commented on code in PR #27224:
URL: https://github.com/apache/datafusion/pull/27224#discussion_r4590054360


##########
datafusion/sqllogictest/test_files/rule_order_join_selection.slt:
##########
@@ -0,0 +1,51 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: See the NOTICE file distributed with this work for 
additional information regarding copyright ownership
+# SPDX-FileContributor: Licensed to the Apache Software Foundation (ASF) under 
one or more contributor license agreements
+
+# This file runs `JoinSelection` after `EnsureRequirements` (see `moved_rule` 
in

Review Comment:
   Is your eventual plan to run the slt tests with arbitrary permutations of 
rules and this one is just a temporary fix?
   
   



##########
datafusion/physical-optimizer/src/ensure_requirements/enforce_distribution.rs:
##########
@@ -806,6 +806,36 @@ fn new_join_conditions(
         .collect()
 }
 
+/// Picks the mode of a hash join still in [`PartitionMode::Auto`].
+///
+/// An `Auto` join declares no distribution requirement, so its inputs would 
get
+/// no repartitioning. `JoinSelection` normally picks the mode first. When it
+/// hasn't (it runs later, or isn't in the rule list), use `Partitioned` if the
+/// join may repartition its inputs and `CollectLeft` otherwise: enforcement 
can
+/// satisfy either one.
+pub(crate) fn resolve_auto_join_mode(

Review Comment:
   What is the final vision for where automatic joins are resolved? Will that 
be part of the "analyzer rules" or would it be during physical planning?



##########
datafusion/sqllogictest/src/test_context.rs:
##########
@@ -345,6 +361,18 @@ impl SchemaProvider for StrictOrdersSchema {
     }
 }
 
+/// A test file that checks a physical optimizer rule gives correct results in
+/// another position: the rule to move, whether it goes after (rather than 
before)
+/// the anchor rule, and the anchor.
+fn moved_rule(file_name: &str) -> Option<(&'static str, bool, &'static str)> {
+    match file_name {
+        "rule_order_join_selection.slt" => {

Review Comment:
   this seems an oddly specific test for just this PR -- I suggest in this case 
we instead use a unit test (I know this is different than my other suggestions) 
 in this PR 
   
   I think a general purpose test would be better (basically that randomly 
permutes the rules and runs the slt, as a type of fuzz tester, for example) 🤔 



##########
datafusion/physical-optimizer/src/ensure_requirements/mod.rs:
##########
@@ -187,8 +188,19 @@ impl PhysicalOptimizerRule for EnsureRequirements {
         // Interleaves are distribution artifacts of Phase 2, which re-derives
         // them from the children's final partitioning. Keeping them would
         // fail as soon as a child loses the partitioning they depend on.
-        use super::enforce_distribution::replace_interleave_with_union;
-        let plan = plan.transform_down(replace_interleave_with_union).data()?;
+        //
+        // A hash join still in `Auto` mode declares no distribution, so it 
would
+        // get no repartitioning. `JoinSelection` normally picks the mode 
earlier;
+        // when it hasn't, pick one here that enforcement can satisfy.
+        use super::enforce_distribution::{

Review Comment:
   I ecommend putting this `use` at the top of the file with the other such 
statements 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to