szehon-ho commented on code in PR #46745:
URL: https://github.com/apache/spark/pull/46745#discussion_r1636908784
##########
docs/sql-performance-tuning.md:
##########
@@ -428,3 +428,118 @@ You can control the details of how AQE works by providing
your own cost evaluato
<td>3.2.0</td>
</tr>
</table>
+
+## Storage Partition Join
+
+Storage Partition Join (SPJ) is an optimization technique in Spark SQL that
makes use the existing storage layout to avoid the shuffle phase.
+
+This is a generalization of the concept of Bucket Joins, which is only
applicable for
[bucketed](sql-data-sources-load-save-functions.html#bucketing-sorting-and-partitioning)
tables, to tables partitioned by functions registered in FunctionCatalog.
Storage Partition Joins are currently supported for compatible V2 DataSources.
Review Comment:
Hm, i built the site and it seems to work, but let me know if another one is
better here?
##########
docs/sql-performance-tuning.md:
##########
@@ -428,3 +428,118 @@ You can control the details of how AQE works by providing
your own cost evaluato
<td>3.2.0</td>
</tr>
</table>
+
+## Storage Partition Join
+
+Storage Partition Join (SPJ) is an optimization technique in Spark SQL that
makes use the existing storage layout to avoid the shuffle phase.
+
+This is a generalization of the concept of Bucket Joins, which is only
applicable for
[bucketed](sql-data-sources-load-save-functions.html#bucketing-sorting-and-partitioning)
tables, to tables partitioned by functions registered in FunctionCatalog.
Storage Partition Joins are currently supported for compatible V2 DataSources.
+
+The following SQL properties enable Storage Partition Join.
Review Comment:
done, added 'in different join queries with various optimizations.' (as
some flags are about different scenarios)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
