This is an automated email from the ASF dual-hosted git repository.
viirya pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 46a28db5 fix: Document enabling comet explain plan usage in Spark
(4.0) (#1176)
46a28db5 is described below
commit 46a28db5e5014fc7fabe8eafc7dbeb5027974c47
Author: Parth Chandra <[email protected]>
AuthorDate: Tue Dec 17 15:50:36 2024 -0800
fix: Document enabling comet explain plan usage in Spark (4.0) (#1176)
---
docs/source/user-guide/tuning.md | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/docs/source/user-guide/tuning.md b/docs/source/user-guide/tuning.md
index af722494..f10a0dde 100644
--- a/docs/source/user-guide/tuning.md
+++ b/docs/source/user-guide/tuning.md
@@ -127,3 +127,13 @@ Here is a guide to some of the native metrics.
| `elapsed_compute` | Total time spent in this operator, fetching batches from
a JVM iterator. |
| `jvm_fetch_time` | Time spent in the JVM fetching input batches to be read
by this `ScanExec` instance. |
| `arrow_ffi_time` | Time spent using Arrow FFI to create Arrow batches from
the memory addresses returned from the JVM. |
+
+## Explain Plan
+### Extended Explain
+With Spark 4.0.0 and newer, Comet can provide extended explain plan
information in the Spark UI. Currently this lists
+reasons why Comet may not have been enabled for specific operations.
+To enable this, in the Spark configuration, set the following:
+```shell
+-c spark.sql.extendedExplainProviders=org.apache.comet.ExtendedExplainInfo
+```
+This will add a section to the detailed plan displayed in the Spark SQL UI
page.
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
