This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 2a51242b1bdb [MINOR][DOCS][SQL] Fix doc comment for
coalescePartitions.parallelismFirst on sql-performance-tuning page
2a51242b1bdb is described below
commit 2a51242b1bdb4a1d469fc81dc0d21c4cde305753
Author: Emil Ejbyfeldt <[email protected]>
AuthorDate: Mon Mar 11 18:14:07 2024 +0800
[MINOR][DOCS][SQL] Fix doc comment for coalescePartitions.parallelismFirst
on sql-performance-tuning page
We missed that this doc comment is duplicated while fixing it in
https://github.com/apache/spark/pull/45437
### What changes were proposed in this pull request?
Documentation fix.
### Why are the changes needed?
Current doc is wrong.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
N/A
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45458 from eejbyfeldt/fix-minor-doc-comment-on-parallism-first.
Authored-by: Emil Ejbyfeldt <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
---
docs/sql-performance-tuning.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/sql-performance-tuning.md b/docs/sql-performance-tuning.md
index 3f523b48e252..b443e3d9c5f5 100644
--- a/docs/sql-performance-tuning.md
+++ b/docs/sql-performance-tuning.md
@@ -278,7 +278,7 @@ This feature coalesces the post shuffle partitions based on
the map output stati
<td><code>spark.sql.adaptive.coalescePartitions.parallelismFirst</code></td>
<td>true</td>
<td>
- When true, Spark ignores the target size specified by
<code>spark.sql.adaptive.advisoryPartitionSizeInBytes</code> (default 64MB)
when coalescing contiguous shuffle partitions, and only respect the minimum
partition size specified by
<code>spark.sql.adaptive.coalescePartitions.minPartitionSize</code> (default
1MB), to maximize the parallelism. This is to avoid performance regressions
when enabling adaptive query execution. It's recommended to set this config to
true on a busy clus [...]
+ When true, Spark ignores the target size specified by
<code>spark.sql.adaptive.advisoryPartitionSizeInBytes</code> (default 64MB)
when coalescing contiguous shuffle partitions, and only respect the minimum
partition size specified by
<code>spark.sql.adaptive.coalescePartitions.minPartitionSize</code> (default
1MB), to maximize the parallelism. This is to avoid performance regressions
when enabling adaptive query execution. It's recommended to set this config to
false on a busy clu [...]
</td>
<td>3.2.0</td>
</tr>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
