This is an automated email from the ASF dual-hosted git repository.
felixybw pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new b43c942004 [VL] Update document, remove the experimental word for
spark.gluten.enabled (#8615)
b43c942004 is described below
commit b43c942004c4da4a35cfe267cb44e249c0183ce2
Author: BInwei Yang <[email protected]>
AuthorDate: Wed Jan 29 12:25:45 2025 -0800
[VL] Update document, remove the experimental word for spark.gluten.enabled
(#8615)
spark.gluten.enabled is stable for long time. It can enable/disable gluten
at run time. With it disable, all query plans fallback to vanilla spark.
Compare to plugin setting, the config may have a little overhead when
generating query plan.
---
docs/Configuration.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/Configuration.md b/docs/Configuration.md
index e217be45ff..d577f9de24 100644
--- a/docs/Configuration.md
+++ b/docs/Configuration.md
@@ -21,7 +21,7 @@ You can add these configurations into spark-defaults.conf to
enable or disable t
| spark.sql.join.preferSortMergeJoin | When true,
prefer sort merge join over shuffled hash join. <br /> Note: Please turn off
preferSortMergeJoin.
[...]
| spark.plugins | To load
Gluten's components by Spark's plug-in loader
[...]
| spark.shuffle.manager | To turn on
Gluten Columnar Shuffle Plugin
[...]
-| spark.gluten.enabled | Enable
Gluten, default is true. Just an experimental property. Recommend to
enable/disable Gluten through the setting for `spark.plugins`.
[...]
+| spark.gluten.enabled | Enable Gluten
at runtime, default is true. It fallbacks to vanilla Spark for all query plans
if set to false. Recommend to enable/disable Gluten through the setting for
`spark.plugins`.
[...]
| spark.gluten.memory.isolation |
(Experimental) Enable isolated memory mode. If true, Gluten controls the
maximum off-heap memory can be used by each task to X, X = executor memory /
max task slots. It's recommended to set true if Gluten serves concurrent
queries within a single session, since not all memory Gluten allocated is
guaranteed to be spillable. In the case, the feature should be enabled to avoid
OOM. Note when true, setting spark.memory.storageF [...]
| spark.gluten.ras.enabled | Enables RAS
(relation algebra selector) during physical planning to generate more efficient
query plan. Note, this feature doesn't bring performance profits by default.
Try exploring option `spark.gluten.ras.costModel` for advanced usage.
[...]
| spark.gluten.sql.columnar.maxBatchSize | Number of
rows to be processed in each batch. Default value is 4096.
[...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
