This is an automated email from the ASF dual-hosted git repository. felixybw pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 2529c2e00 Improve the configuration document (#7334)
2529c2e00 is described below
commit 2529c2e00dfbf386901485b37de6793dde432382
Author: Jiaan Geng <[email protected]>
AuthorDate: Wed Sep 25 03:14:05 2024 +0800
Improve the configuration document (#7334)
---
docs/Configuration.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/docs/Configuration.md b/docs/Configuration.md
index 4983f92e2..e044caf85 100644
--- a/docs/Configuration.md
+++ b/docs/Configuration.md
@@ -13,12 +13,12 @@ You can add these configurations into spark-defaults.conf
to enable or disable t
| Parameters | Description
[...]
|--------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
-| spark.driver.extraClassPath | To add Gluten
Plugin jar file in Spark Driver
[...]
-| spark.executor.extraClassPath | To add Gluten
Plugin jar file in Spark Executor
[...]
-| spark.executor.memory | To set up how
much memory to be used for Spark Executor.
[...]
-| spark.memory.offHeap.size | To set up how
much memory to be used for Java OffHeap.<br /> Please notice Gluten Plugin will
leverage this setting to allocate memory space for native usage even offHeap is
disabled. <br /> The value is based on your system and it is recommended to set
it larger if you are facing Out of Memory issue in Gluten Plugin
[...]
-| spark.sql.sources.useV1SourceList | Choose to use
V1 source
[...]
-| spark.sql.join.preferSortMergeJoin | To turn off
preferSortMergeJoin in Spark
[...]
+| spark.driver.extraClassPath | Gluten Plugin
jar file to prepend to the classpath of the driver.
[...]
+| spark.executor.extraClassPath | Gluten Plugin
jar file to prepend to the classpath of executors.
[...]
+| spark.executor.memory | Amount of
memory to use per executor process, in the same format as JVM memory strings
with a size unit suffix ("k", "m", "g" or "t") (e.g. 512m, 2g).
[...]
+| spark.memory.offHeap.size | The absolute
amount of memory which can be used for off-heap allocation, in bytes unless
otherwise specified.<br /> Note: Gluten Plugin will leverage this setting to
allocate memory space for native usage even offHeap is disabled. <br /> The
value is based on your system and it is recommended to set it larger if you are
facing Out of Memory issue in Gluten Plugin
[...]
+| spark.sql.sources.useV1SourceList | A
comma-separated list of data source short names or fully qualified data source
implementation class names for which Data Source V2 code path is disabled. <br
/> Note: Please use V1 source for avro.
[...]
+| spark.sql.join.preferSortMergeJoin | When true,
prefer sort merge join over shuffled hash join. <br /> Note: Please turn off
preferSortMergeJoin.
[...]
| spark.plugins | To load
Gluten's components by Spark's plug-in loader
[...]
| spark.shuffle.manager | To turn on
Gluten Columnar Shuffle Plugin
[...]
| spark.gluten.enabled | Enable
Gluten, default is true. Just an experimental property. Recommend to
enable/disable Gluten through the setting for `spark.plugins`.
[...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
