(spark) branch master updated: [SPARK-53098][SQL][TEST-ONLY][FOLLOWUP] Fix a test when returning single-pass result

[wenchen]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new a09ea772e374 [SPARK-53098][SQL][TEST-ONLY][FOLLOWUP] Fix a test when 
returning single-pass result
a09ea772e374 is described below

commit a09ea772e37408a967f190719f3974cbb7008cbc
Author: Mihailo Timotic <mihailo.timo...@databricks.com>
AuthorDate: Thu Aug 7 07:56:18 2025 +0800

    [SPARK-53098][SQL][TEST-ONLY][FOLLOWUP] Fix a test when returning 
single-pass result
    
    ### What changes were proposed in this pull request?
    Fix a test recently added here: https://github.com/apache/spark/pull/51812
    
    ### Why are the changes needed?
    Single-pass always resolved the self-join condition to the left branch, 
regardless of conf values. Therefore, we need to disable returning single-pass 
result when conf is false
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Fixed a test
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #51872 from mihailotim-db/mihailotim-db/fix_test.
    
    Authored-by: Mihailo Timotic <mihailo.timo...@databricks.com>
    Signed-off-by: Wenchen Fan <wenc...@databricks.com>
---
 .../src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala
index c3c43273907a..bc603f111433 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala
@@ -510,7 +510,10 @@ class DataFrameSelfJoinSuite extends QueryTest with 
SharedSparkSession {
         // Disable auto-resolution of ambiguity because we want to test 
behavior before
         // `resolveSelfJoinCondition` fully kicks in (while we still have 
ambiguous join condition)
         SQLConf.DATAFRAME_SELF_JOIN_AUTO_RESOLVE_AMBIGUITY.key -> "false",
-        SQLConf.DONT_DEDUPLICATE_EXPRESSION_IF_EXPR_ID_IN_OUTPUT.key -> 
conf.toString
+        SQLConf.DONT_DEDUPLICATE_EXPRESSION_IF_EXPR_ID_IN_OUTPUT.key -> 
conf.toString,
+        // Single-pass analyzer always maps self-join condition to the left 
branch, regardless of
+        // conf so we return single-pass result only if deduplication conf is 
true.
+        SQLConf.ANALYZER_DUAL_RUN_RETURN_SINGLE_PASS_RESULT.key -> 
conf.toString
       ) {
         val analyzedPlan =
           df1.join(df3, df1.col("key") === df3.col("key"), 
"left_outer").queryExecution.analyzed


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org