This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 265363d [SPARK-33451][DOCS] Change to
'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes' in documentation
265363d is described below
commit 265363d89e8ffc452e100289f8f1a94cd9d6cc44
Author: aof00 <[email protected]>
AuthorDate: Mon Nov 16 10:32:00 2020 +0900
[SPARK-33451][DOCS] Change to
'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes' in documentation
### What changes were proposed in this pull request?
In the 'Optimizing Skew Join' section of the following two pages:
1.
[https://spark.apache.org/docs/3.0.0/sql-performance-tuning.html](https://spark.apache.org/docs/3.0.0/sql-performance-tuning.html)
2.
[https://spark.apache.org/docs/3.0.1/sql-performance-tuning.html](https://spark.apache.org/docs/3.0.1/sql-performance-tuning.html)
The configuration 'spark.sql.adaptive.skewedPartitionThresholdInBytes'
should be changed to
'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes', The former is
missing the 'skewJoin'.
### Why are the changes needed?
To document the correct name of configuration
### Does this PR introduce _any_ user-facing change?
Yes, this is a user-facing doc change.
### How was this patch tested?
Jenkins / CI builds in this PR.
Closes #30376 from aof00/doc_change.
Authored-by: aof00 <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit 0933f1c6c27c7a087c8dbe485bd08371ce5f5695)
Signed-off-by: HyukjinKwon <[email protected]>
---
docs/sql-performance-tuning.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/sql-performance-tuning.md b/docs/sql-performance-tuning.md
index 5d8c3b6..49b32e7 100644
--- a/docs/sql-performance-tuning.md
+++ b/docs/sql-performance-tuning.md
@@ -280,7 +280,7 @@ Data skew can severely downgrade the performance of join
queries. This feature d
<td><code>spark.sql.adaptive.skewJoin.skewedPartitionFactor</code></td>
<td>10</td>
<td>
- A partition is considered as skewed if its size is larger than this
factor multiplying the median partition size and also larger than
<code>spark.sql.adaptive.skewedPartitionThresholdInBytes</code>.
+ A partition is considered as skewed if its size is larger than this
factor multiplying the median partition size and also larger than
<code>spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes</code>.
</td>
<td>3.0.0</td>
</tr>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
