pan3793 commented on code in PR #57762:
URL: https://github.com/apache/spark/pull/57762#discussion_r3713617878
##########
docs/sql-performance-tuning.md:
##########
@@ -535,9 +535,9 @@ The following SQL properties enable Storage Partition Join
in different join que
<td><code>spark.sql.requireAllClusterKeysForCoPartition</code></td>
<td>true</td>
<td>
- When true, require the join or MERGE keys to be same and in the same
order as the partition keys to eliminate shuffle. Hence, set to <b>false</b> in
this situation to eliminate shuffle.
+ When true, storage-partitioned join requires every join or MERGE key
to be covered by the partition keys, ignoring key order and duplicated keys, to
eliminate shuffle. When the partition keys cover only part of the join or MERGE
keys, set to <b>false</b> to eliminate shuffle, at the risk of data skew and
reduced parallelism from the coarser storage partitioning.
</td>
- <td>3.4.0</td>
+ <td>3.3.0</td>
Review Comment:
a small doc fix, the config `spark.sql.requireAllClusterKeysForCoPartition`
is introduced in 3.3.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
