This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new e56859f84 Publish built docs triggered by
3aa3dc707821ea7620c50658eb22af90b6b0edb7
e56859f84 is described below
commit e56859f84cde677129d31389dde288d021585686
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Wed Jun 18 00:18:28 2025 +0000
Publish built docs triggered by 3aa3dc707821ea7620c50658eb22af90b6b0edb7
---
_sources/user-guide/configs.md.txt | 2 ++
searchindex.js | 2 +-
user-guide/configs.html | 8 ++++++++
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/configs.md.txt
b/_sources/user-guide/configs.md.txt
index f27830320..acd1e717c 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -75,6 +75,8 @@ Comet provides the following configuration settings.
| spark.comet.memory.overhead.min | Minimum amount of additional memory to be
allocated per executor process for Comet, in MiB, when running Spark in on-heap
mode. For more information, refer to the Comet Tuning Guide
(https://datafusion.apache.org/comet/user-guide/tuning.html). | 402653184b |
| spark.comet.memoryOverhead | The amount of additional memory to be allocated
per executor process for Comet, in MiB, when running Spark in on-heap mode.
This config is optional. If this is not specified, it will be set to
`spark.comet.memory.overhead.factor` * `spark.executor.memory`. For more
information, refer to the Comet Tuning Guide
(https://datafusion.apache.org/comet/user-guide/tuning.html). | |
| spark.comet.metrics.updateInterval | The interval in milliseconds to update
metrics. If interval is negative, metrics will be updated upon task completion.
| 3000 |
+| spark.comet.native.shuffle.partitioning.hash.enabled | Whether to enable
hash partitioning for Comet native shuffle. | true |
+| spark.comet.native.shuffle.partitioning.range.enabled | Whether to enable
range partitioning for Comet native shuffle. | true |
| spark.comet.nativeLoadRequired | Whether to require Comet native library to
load successfully when Comet is enabled. If not, Comet will silently fallback
to Spark when it fails to load the native lib. Otherwise, an error will be
thrown and the Spark job will be aborted. | false |
| spark.comet.parquet.enable.directBuffer | Whether to use Java direct byte
buffer when reading Parquet. | false |
| spark.comet.parquet.read.io.adjust.readRange.skew | In the parallel reader,
if the read ranges submitted are skewed in sizes, this option will cause the
reader to break up larger read ranges into smaller ranges to reduce the skew.
This will result in a slightly larger number of connections opened to the file
system but may give improved performance. | false |
diff --git a/searchindex.js b/searchindex.js
index d8845b88c..a2a6347b1 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]],
"2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[14,
"ansi-mode"]], "API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[13, null]],
"Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[19,
null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]],
"2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[14,
"ansi-mode"]], "API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[13, null]],
"Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[19,
null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
diff --git a/user-guide/configs.html b/user-guide/configs.html
index cdd589429..961c754d5 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -532,6 +532,14 @@ TO MODIFY THIS CONTENT MAKE SURE THAT YOU MAKE YOUR
CHANGES TO THE TEMPLATE FILE
<td><p>The interval in milliseconds to update metrics. If interval is
negative, metrics will be updated upon task completion.</p></td>
<td><p>3000</p></td>
</tr>
+<tr
class="row-even"><td><p>spark.comet.native.shuffle.partitioning.hash.enabled</p></td>
+<td><p>Whether to enable hash partitioning for Comet native shuffle.</p></td>
+<td><p>true</p></td>
+</tr>
+<tr
class="row-odd"><td><p>spark.comet.native.shuffle.partitioning.range.enabled</p></td>
+<td><p>Whether to enable range partitioning for Comet native shuffle.</p></td>
+<td><p>true</p></td>
+</tr>
<tr class="row-even"><td><p>spark.comet.nativeLoadRequired</p></td>
<td><p>Whether to require Comet native library to load successfully when Comet
is enabled. If not, Comet will silently fallback to Spark when it fails to load
the native lib. Otherwise, an error will be thrown and the Spark job will be
aborted.</p></td>
<td><p>false</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@datafusion.apache.org
For additional commands, e-mail: commits-h...@datafusion.apache.org
