This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new ff05015ec Publish built docs triggered by
eaee24cba176f74b9f3e528f0b451c7b88129593
ff05015ec is described below
commit ff05015ec5ef784126f93caf456a1cc8ce473524
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Nov 7 21:49:00 2025 +0000
Publish built docs triggered by eaee24cba176f74b9f3e528f0b451c7b88129593
---
_sources/user-guide/latest/configs.md.txt | 8 ++++----
searchindex.js | 2 +-
user-guide/latest/configs.html | 8 ++++----
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/_sources/user-guide/latest/configs.md.txt
b/_sources/user-guide/latest/configs.md.txt
index 6caaa53b1..66e1795cb 100644
--- a/_sources/user-guide/latest/configs.md.txt
+++ b/_sources/user-guide/latest/configs.md.txt
@@ -63,7 +63,7 @@ Comet provides the following configuration settings.
| `spark.comet.caseConversion.enabled` | Java uses locale-specific rules when
converting strings to upper or lower case and Rust does not, so we disable
upper and lower by default. | false |
| `spark.comet.debug.enabled` | Whether to enable debug mode for Comet. When
enabled, Comet will do additional checks for debugging purpose. For example,
validating array when importing arrays from JVM at native side. Note that these
checks may be expensive in performance and should only be enabled for debugging
purpose. | false |
| `spark.comet.dppFallback.enabled` | Whether to fall back to Spark for
queries that use DPP. | true |
-| `spark.comet.enabled` | Whether to enable Comet extension for Spark. When
this is turned on, Spark will use Comet to read Parquet data source. Note that
to enable native vectorized execution, both this config and
`spark.comet.exec.enabled` need to be enabled. By default, this config is the
value of the env var `ENABLE_COMET` if set, or true otherwise. | true |
+| `spark.comet.enabled` | Whether to enable Comet extension for Spark. When
this is turned on, Spark will use Comet to read Parquet data source. Note that
to enable native vectorized execution, both this config and
`spark.comet.exec.enabled` need to be enabled. Can be overridden by environment
variable `ENABLE_COMET`. | true |
| `spark.comet.exceptionOnDatetimeRebase` | Whether to throw exception when
seeing dates/timestamps from the legacy hybrid (Julian + Gregorian) calendar.
Since Spark 3, dates/timestamps were written according to the Proleptic
Gregorian calendar. When this is true, Comet will throw exceptions when seeing
these dates/timestamps that were written by Spark version before 3.0. If this
is false, these dates/timestamps will be read as if they were written to the
Proleptic Gregorian calendar and [...]
| `spark.comet.exec.enabled` | Whether to enable Comet native vectorized
execution for Spark. This controls whether Spark should convert operators into
their Comet counterparts and execute them in native space. Note: each operator
is associated with a separate config in the format of
`spark.comet.exec.<operator_name>.enabled` at the moment, and both the config
and this need to be turned on, in order for the operator to be executed in
native. | true |
| `spark.comet.exec.replaceSortMergeJoin` | Experimental feature to force
Spark to replace SortMergeJoin with ShuffledHashJoin for improved performance.
This feature is not stable yet. For more information, refer to the [Comet
Tuning Guide](https://datafusion.apache.org/comet/user-guide/tuning.html). |
false |
@@ -86,7 +86,7 @@ These settings can be used to determine which parts of the
plan are accelerated
| `spark.comet.explain.rules` | When this setting is enabled, Comet will log
all plan transformations performed in physical optimizer rules. Default: false
| false |
| `spark.comet.explain.verbose.enabled` | When this setting is enabled,
Comet's extended explain output will provide the full query plan annotated with
fallback reasons as well as a summary of how much of the plan was accelerated
by Comet. When this setting is disabled, a list of fallback reasons will be
provided instead. | false |
| `spark.comet.explainFallback.enabled` | When this setting is enabled, Comet
will provide logging explaining the reason(s) why a query stage cannot be
executed natively. Set this to false to reduce the amount of logging. | false |
-| `spark.comet.logFallbackReasons.enabled` | When this setting is enabled,
Comet will log warnings for all fallback reasons. | false |
+| `spark.comet.logFallbackReasons.enabled` | When this setting is enabled,
Comet will log warnings for all fallback reasons. Can be overridden by
environment variable `ENABLE_COMET_LOG_FALLBACK_REASONS`. | false |
<!--END:CONFIG_TABLE-->
## Shuffle Configuration Settings
@@ -127,10 +127,10 @@ These settings can be used to determine which parts of
the plan are accelerated
| Config | Description | Default Value |
|--------|-------------|---------------|
| `spark.comet.columnar.shuffle.memory.factor` | Fraction of Comet memory to
be allocated per executor process for columnar shuffle when running in on-heap
mode. For more information, refer to the [Comet Tuning
Guide](https://datafusion.apache.org/comet/user-guide/tuning.html). | 1.0 |
-| `spark.comet.exec.onHeap.enabled` | Whether to allow Comet to run in on-heap
mode. Required for running Spark SQL tests. | false |
+| `spark.comet.exec.onHeap.enabled` | Whether to allow Comet to run in on-heap
mode. Required for running Spark SQL tests. Can be overridden by environment
variable `ENABLE_COMET_ONHEAP`. | false |
| `spark.comet.exec.onHeap.memoryPool` | The type of memory pool to be used
for Comet native execution when running Spark in on-heap mode. Available pool
types are `greedy`, `fair_spill`, `greedy_task_shared`,
`fair_spill_task_shared`, `greedy_global`, `fair_spill_global`, and
`unbounded`. | greedy_task_shared |
| `spark.comet.memoryOverhead` | The amount of additional memory to be
allocated per executor process for Comet, in MiB, when running Spark in on-heap
mode. | 1024 MiB |
-| `spark.comet.testing.strict` | Experimental option to enable strict testing,
which will fail tests that could be more comprehensive, such as checking for a
specific fallback reason | false |
+| `spark.comet.testing.strict` | Experimental option to enable strict testing,
which will fail tests that could be more comprehensive, such as checking for a
specific fallback reason. Can be overridden by environment variable
`ENABLE_COMET_STRICT_TESTING`. | false |
<!--END:CONFIG_TABLE-->
## Enabling or Disabling Individual Operators
diff --git a/searchindex.js b/searchindex.js
index f5652c129..e6c34d08b 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[18, "install-comet"]],
"2. Clone Spark and Apply Diff": [[18, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[18, "run-spark-sql-tests"]], "ANSI Mode": [[21,
"ansi-mode"], [34, "ansi-mode"], [74, "ansi-mode"]], "ANSI mode": [[47,
"ansi-mode"], [60, "ansi-mode"]], "API Differences Between Spark Versions":
[[3, "api-differences-between-spark-versions"]], "ASF Links": [[2, null], [2,
null]], "Accelerating Apache Iceberg Parque [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[18, "install-comet"]],
"2. Clone Spark and Apply Diff": [[18, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[18, "run-spark-sql-tests"]], "ANSI Mode": [[21,
"ansi-mode"], [34, "ansi-mode"], [74, "ansi-mode"]], "ANSI mode": [[47,
"ansi-mode"], [60, "ansi-mode"]], "API Differences Between Spark Versions":
[[3, "api-differences-between-spark-versions"]], "ASF Links": [[2, null], [2,
null]], "Accelerating Apache Iceberg Parque [...]
\ No newline at end of file
diff --git a/user-guide/latest/configs.html b/user-guide/latest/configs.html
index 4c09ee797..1ea17ef23 100644
--- a/user-guide/latest/configs.html
+++ b/user-guide/latest/configs.html
@@ -588,7 +588,7 @@ under the License.
<td><p>true</p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.enabled</span></code></p></td>
-<td><p>Whether to enable Comet extension for Spark. When this is turned on,
Spark will use Comet to read Parquet data source. Note that to enable native
vectorized execution, both this config and <code class="docutils literal
notranslate"><span class="pre">spark.comet.exec.enabled</span></code> need to
be enabled. By default, this config is the value of the env var <code
class="docutils literal notranslate"><span
class="pre">ENABLE_COMET</span></code> if set, or true otherwise.</p></td>
+<td><p>Whether to enable Comet extension for Spark. When this is turned on,
Spark will use Comet to read Parquet data source. Note that to enable native
vectorized execution, both this config and <code class="docutils literal
notranslate"><span class="pre">spark.comet.exec.enabled</span></code> need to
be enabled. Can be overridden by environment variable <code class="docutils
literal notranslate"><span class="pre">ENABLE_COMET</span></code>.</p></td>
<td><p>true</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.exceptionOnDatetimeRebase</span></code></p></td>
@@ -658,7 +658,7 @@ under the License.
<td><p>false</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.logFallbackReasons.enabled</span></code></p></td>
-<td><p>When this setting is enabled, Comet will log warnings for all fallback
reasons.</p></td>
+<td><p>When this setting is enabled, Comet will log warnings for all fallback
reasons. Can be overridden by environment variable <code class="docutils
literal notranslate"><span
class="pre">ENABLE_COMET_LOG_FALLBACK_REASONS</span></code>.</p></td>
<td><p>false</p></td>
</tr>
</tbody>
@@ -777,7 +777,7 @@ under the License.
<td><p>1.0</p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.exec.onHeap.enabled</span></code></p></td>
-<td><p>Whether to allow Comet to run in on-heap mode. Required for running
Spark SQL tests.</p></td>
+<td><p>Whether to allow Comet to run in on-heap mode. Required for running
Spark SQL tests. Can be overridden by environment variable <code
class="docutils literal notranslate"><span
class="pre">ENABLE_COMET_ONHEAP</span></code>.</p></td>
<td><p>false</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.exec.onHeap.memoryPool</span></code></p></td>
@@ -789,7 +789,7 @@ under the License.
<td><p>1024 MiB</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.testing.strict</span></code></p></td>
-<td><p>Experimental option to enable strict testing, which will fail tests
that could be more comprehensive, such as checking for a specific fallback
reason</p></td>
+<td><p>Experimental option to enable strict testing, which will fail tests
that could be more comprehensive, such as checking for a specific fallback
reason. Can be overridden by environment variable <code class="docutils literal
notranslate"><span
class="pre">ENABLE_COMET_STRICT_TESTING</span></code>.</p></td>
<td><p>false</p></td>
</tr>
</tbody>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
