This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 0ae10736d6 Publish built docs triggered by
6acfe7dee1b4bff77f2c9e840efa9cd04777c41f
0ae10736d6 is described below
commit 0ae10736d67225e8fd1264228806c2b6f845ab20
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Mon Aug 3 19:41:55 2026 +0000
Publish built docs triggered by 6acfe7dee1b4bff77f2c9e840efa9cd04777c41f
---
.../latest/understanding-comet-plans.md.txt | 21 ++++++++++++++++++---
searchindex.js | 2 +-
user-guide/latest/understanding-comet-plans.html | 20 +++++++++++++++++---
3 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/_sources/user-guide/latest/understanding-comet-plans.md.txt
b/_sources/user-guide/latest/understanding-comet-plans.md.txt
index 130762d560..2a6945c947 100644
--- a/_sources/user-guide/latest/understanding-comet-plans.md.txt
+++ b/_sources/user-guide/latest/understanding-comet-plans.md.txt
@@ -144,7 +144,7 @@ CometColumnarToRow
+- CometProject [COMET-INFO: JVM codegen dispatcher: hypot, levenshtein]
+- CometNativeScan parquet spark_catalog.default.t
-Comet accelerated 2 out of 2 eligible operators (100%). Final plan contains 1
transitions between Spark and Comet.
+Comet accelerated 2 out of 2 eligible operators (100%). Final plan contains 1
transitions between Spark and Comet. Accelerated expressions: 0 native, 2
codegen dispatch.
```
Note that the operator is still `CometProject` (Comet-accelerated); only the
@@ -165,7 +165,12 @@ The Spark SQL UI then shows an additional section under
the detailed plan.
The format is controlled by `spark.comet.explain.format`:
- `verbose` (default): the full plan annotated with fallback reasons, plus a
- summary of how much of the plan is accelerated.
+ summary of how much of the plan is accelerated. The summary reports operator
+ coverage, the number of Spark/Comet transitions, and how many distinct
+ expressions Comet accelerated - split into those lowered to native DataFusion
+ expressions and those routed through the JVM codegen dispatcher. Expression
+ names are counted once per plan, and structural nodes (attribute references,
+ literals, aliases) are excluded.
- `fallback`: a list of fallback reasons only.
This is the most convenient option on Spark 4.0 because the output is shown
@@ -201,12 +206,22 @@ val info = new ExtendedExplainInfo()
// Sorted, deduplicated list of fallback reasons across the whole plan.
val reasons: Seq[String] = info.getFallbackReasons(plan)
+// Sorted, deduplicated names of the expressions Comet accelerated, split by
how
+// they run. Structural nodes (attribute references, literals, aliases) are not
+// reported.
+val native: Seq[String] = info.getNativeExpressions(plan)
+val dispatched: Seq[String] = info.getCodegenDispatchExpressions(plan)
+
// Formatted string. Honors spark.comet.explain.format:
// - "verbose" -> the full plan annotated with per-node fallback reasons
// - "fallback" -> just the list of reasons
val formatted: String = info.generateExtendedInfo(plan)
```
+A name can appear in both lists: the same function may be lowered natively for
+one set of arguments and routed through the dispatcher for another elsewhere in
+the same plan.
+
Example:
```
@@ -224,7 +239,7 @@ Project [COMET: from_unixtime(eventTime#5L, yyyy-MM-dd
HH:mm:ss, Some(GMT)) is n
+- CometColumnarToRow
+- CometNativeScan parquet
-Comet accelerated 1 out of 2 eligible operators (50%). Final plan contains 1
transitions between Spark and Comet.
+Comet accelerated 1 out of 2 eligible operators (50%). Final plan contains 1
transitions between Spark and Comet. Accelerated expressions: 0 native, 0
codegen dispatch.
```
## Comet Operator Reference
diff --git a/searchindex.js b/searchindex.js
index 4132c65630..2fd0a49ba0 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[54, "id1"]], "%": [[52, "id1"]], "&":
[[42, "id1"]], "*": [[52, "id2"]], "+": [[52, "id3"]], "-": [[52, "id4"]], "/":
[[52, "id5"]], "1. Format Your Code": [[39, "format-your-code"]], "1. Install
Comet": [[60, "install-comet"], [70, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[27, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[39, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[60,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[54, "id1"]], "%": [[52, "id1"]], "&":
[[42, "id1"]], "*": [[52, "id2"]], "+": [[52, "id3"]], "-": [[52, "id4"]], "/":
[[52, "id5"]], "1. Format Your Code": [[39, "format-your-code"]], "1. Install
Comet": [[60, "install-comet"], [70, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[27, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[39, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[60,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/understanding-comet-plans.html
b/user-guide/latest/understanding-comet-plans.html
index 5698e68a44..a285017a7b 100644
--- a/user-guide/latest/understanding-comet-plans.html
+++ b/user-guide/latest/understanding-comet-plans.html
@@ -882,7 +882,7 @@ expression ran here” marker.</p>
<span class="o">+-</span> <span class="n">CometProject</span> <span
class="p">[</span><span class="n">COMET</span><span class="o">-</span><span
class="n">INFO</span><span class="p">:</span> <span class="n">JVM</span> <span
class="n">codegen</span> <span class="n">dispatcher</span><span
class="p">:</span> <span class="n">hypot</span><span class="p">,</span> <span
class="n">levenshtein</span><span class="p">]</span>
<span class="o">+-</span> <span class="n">CometNativeScan</span> <span
class="n">parquet</span> <span class="n">spark_catalog</span><span
class="o">.</span><span class="n">default</span><span class="o">.</span><span
class="n">t</span>
-<span class="n">Comet</span> <span class="n">accelerated</span> <span
class="mi">2</span> <span class="n">out</span> <span class="n">of</span> <span
class="mi">2</span> <span class="n">eligible</span> <span
class="n">operators</span> <span class="p">(</span><span
class="mi">100</span><span class="o">%</span><span class="p">)</span><span
class="o">.</span> <span class="n">Final</span> <span class="n">plan</span>
<span class="n">contains</span> <span class="mi">1</span> <span class="n">tra
[...]
+<span class="n">Comet</span> <span class="n">accelerated</span> <span
class="mi">2</span> <span class="n">out</span> <span class="n">of</span> <span
class="mi">2</span> <span class="n">eligible</span> <span
class="n">operators</span> <span class="p">(</span><span
class="mi">100</span><span class="o">%</span><span class="p">)</span><span
class="o">.</span> <span class="n">Final</span> <span class="n">plan</span>
<span class="n">contains</span> <span class="mi">1</span> <span class="n">tra
[...]
</pre></div>
</div>
<p>Note that the operator is still <code class="docutils literal
notranslate"><span class="pre">CometProject</span></code> (Comet-accelerated);
only the
@@ -901,7 +901,12 @@ loads via the <code class="docutils literal
notranslate"><span class="pre">spark
The format is controlled by <code class="docutils literal notranslate"><span
class="pre">spark.comet.explain.format</span></code>:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span
class="pre">verbose</span></code> (default): the full plan annotated with
fallback reasons, plus a
-summary of how much of the plan is accelerated.</p></li>
+summary of how much of the plan is accelerated. The summary reports operator
+coverage, the number of Spark/Comet transitions, and how many distinct
+expressions Comet accelerated - split into those lowered to native DataFusion
+expressions and those routed through the JVM codegen dispatcher. Expression
+names are counted once per plan, and structural nodes (attribute references,
+literals, aliases) are excluded.</p></li>
<li><p><code class="docutils literal notranslate"><span
class="pre">fallback</span></code>: a list of fallback reasons only.</p></li>
</ul>
<p>This is the most convenient option on Spark 4.0 because the output is shown
@@ -936,12 +941,21 @@ it works on any Spark version because it does not rely on
the Spark 4.0
<span class="c1">// Sorted, deduplicated list of fallback reasons across the
whole plan.</span>
<span class="kd">val</span><span class="w"> </span><span
class="n">reasons</span><span class="p">:</span><span class="w"> </span><span
class="nc">Seq</span><span class="p">[</span><span
class="nc">String</span><span class="p">]</span><span class="w"> </span><span
class="o">=</span><span class="w"> </span><span class="n">info</span><span
class="p">.</span><span class="n">getFallbackReasons</span><span
class="p">(</span><span class="n">plan</span><span class="p">)</span>
+<span class="c1">// Sorted, deduplicated names of the expressions Comet
accelerated, split by how</span>
+<span class="c1">// they run. Structural nodes (attribute references,
literals, aliases) are not</span>
+<span class="c1">// reported.</span>
+<span class="kd">val</span><span class="w"> </span><span
class="n">native</span><span class="p">:</span><span class="w"> </span><span
class="nc">Seq</span><span class="p">[</span><span
class="nc">String</span><span class="p">]</span><span class="w"> </span><span
class="o">=</span><span class="w"> </span><span class="n">info</span><span
class="p">.</span><span class="n">getNativeExpressions</span><span
class="p">(</span><span class="n">plan</span><span class="p">)</span>
+<span class="kd">val</span><span class="w"> </span><span
class="n">dispatched</span><span class="p">:</span><span class="w">
</span><span class="nc">Seq</span><span class="p">[</span><span
class="nc">String</span><span class="p">]</span><span class="w"> </span><span
class="o">=</span><span class="w"> </span><span class="n">info</span><span
class="p">.</span><span class="n">getCodegenDispatchExpressions</span><span
class="p">(</span><span class="n">plan</span><span class="p">)</span>
+
<span class="c1">// Formatted string. Honors spark.comet.explain.format:</span>
<span class="c1">// - "verbose" -> the full plan annotated
with per-node fallback reasons</span>
<span class="c1">// - "fallback" -> just the list of
reasons</span>
<span class="kd">val</span><span class="w"> </span><span
class="n">formatted</span><span class="p">:</span><span class="w"> </span><span
class="nc">String</span><span class="w"> </span><span class="o">=</span><span
class="w"> </span><span class="n">info</span><span class="p">.</span><span
class="n">generateExtendedInfo</span><span class="p">(</span><span
class="n">plan</span><span class="p">)</span>
</pre></div>
</div>
+<p>A name can appear in both lists: the same function may be lowered natively
for
+one set of arguments and routed through the dispatcher for another elsewhere in
+the same plan.</p>
<p>Example:</p>
<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">val</span> <span
class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span
class="o">.</span><span class="n">sql</span><span class="p">(</span><span
class="s2">"SELECT from_unixtime(eventTime) FROM events"</span><span
class="p">)</span>
@@ -955,7 +969,7 @@ it works on any Spark version because it does not rely on
the Spark 4.0
<span class="o">+-</span> <span class="n">CometColumnarToRow</span>
<span class="o">+-</span> <span class="n">CometNativeScan</span> <span
class="n">parquet</span>
-<span class="n">Comet</span> <span class="n">accelerated</span> <span
class="mi">1</span> <span class="n">out</span> <span class="n">of</span> <span
class="mi">2</span> <span class="n">eligible</span> <span
class="n">operators</span> <span class="p">(</span><span
class="mi">50</span><span class="o">%</span><span class="p">)</span><span
class="o">.</span> <span class="n">Final</span> <span class="n">plan</span>
<span class="n">contains</span> <span class="mi">1</span> <span class="n">tran
[...]
+<span class="n">Comet</span> <span class="n">accelerated</span> <span
class="mi">1</span> <span class="n">out</span> <span class="n">of</span> <span
class="mi">2</span> <span class="n">eligible</span> <span
class="n">operators</span> <span class="p">(</span><span
class="mi">50</span><span class="o">%</span><span class="p">)</span><span
class="o">.</span> <span class="n">Final</span> <span class="n">plan</span>
<span class="n">contains</span> <span class="mi">1</span> <span class="n">tran
[...]
</pre></div>
</div>
</section>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
