[jira] [Created] (SPARK-40715) Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size

[Wan Kun (Jira)]

Wan Kun created SPARK-40715:
-------------------------------

             Summary: Support preferring shuffled hash join thought 
LocalMapThreshold is less than advisory partition size
                 Key: SPARK-40715
                 URL: https://issues.apache.org/jira/browse/SPARK-40715
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.4.0
            Reporter: Wan Kun


Now DynamicJoinSelection will prefer shuffled hash join only if 
*advisoryPartitionSize <= maxShuffledHashJoinLocalMapThreshold*
We can also prefer shuffled hash join if the partitions in one side are smaller 
are smaller than advisoryPartitionSize, and the partitions in the other side 
are all larger enough(larger than advisoryPartitionSize)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org