This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d60104c43d Publish built docs triggered by
57ef3275d3d4e6e3c4d5f3de27b9b71daf736164
d60104c43d is described below
commit d60104c43d577c5e204dba9e9334ef049c7366cf
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Tue Sep 22 00:00:11 2026 +0000
Publish built docs triggered by 57ef3275d3d4e6e3c4d5f3de27b9b71daf736164
---
...aid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg | 2 +-
_sources/contributor-guide/plugin_overview.md.txt | 14 +++++++++-----
contributor-guide/plugin_overview.html | 21 ++++++++++++---------
searchindex.js | 2 +-
4 files changed, 23 insertions(+), 16 deletions(-)
diff --git a/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
b/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
index 5c01043340..fb39632a32 100644
--- a/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
+++ b/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
@@ -1 +1 @@
-<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; class="flowchart" style="max-width:
511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 942"
role="graphics-document document"
aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet
ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes
edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes
dash{to{stroke-dashoffset:0;}}#my-svg [...]
\ No newline at end of file
+<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; class="flowchart" style="max-width:
511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 942"
role="graphics-document document"
aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet
ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes
edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes
dash{to{stroke-dashoffset:0;}}#my-svg [...]
\ No newline at end of file
diff --git a/_sources/contributor-guide/plugin_overview.md.txt
b/_sources/contributor-guide/plugin_overview.md.txt
index c3330a65d6..6dfb0bdf3c 100644
--- a/_sources/contributor-guide/plugin_overview.md.txt
+++ b/_sources/contributor-guide/plugin_overview.md.txt
@@ -47,11 +47,15 @@ The plugin also registers `CometSparkSessionExtensions`
with Spark's extension A
## CometSparkSessionExtensions
-On initialization, this class registers two physical plan optimization rules
with Spark: `CometScanRule`
-and `CometExecRule`. These rules run whenever a query stage is being planned
during Adaptive Query Execution, and
-run once for the entire plan when Adaptive Query Execution is disabled.
+On initialization, this class registers one physical plan optimization rule
with Spark: `CometRule`. It runs whenever
+a query stage is being planned during Adaptive Query Execution, and runs once
for the entire plan when Adaptive Query
+Execution is disabled.
-### CometScanRule
+`CometRule` is two phases, applied in order: scan conversion
(`CometScanRule`), then operator conversion
+(`CometExecRule`). The order matters, because operator conversion builds its
native plan up from the nodes that scan
+conversion produces. Each phase is described below.
+
+### Phase 1: CometScanRule
`CometScanRule` replaces any Parquet scans with Comet operators. There are
different paths for Spark v1 and v2 data sources.
@@ -68,7 +72,7 @@ convert the output from Spark's scan to Arrow arrays. Note
that both `spark.come
Refer to the [Supported Spark Data
Types](https://datafusion.apache.org/comet/user-guide/datatypes.html) section
in the contributor guide to see a list of currently supported data types.
-### CometExecRule
+### Phase 2: CometExecRule
This rule traverses bottom-up from the original Spark plan and attempts to
replace each operator with a Comet equivalent.
For example, a `ProjectExec` will be replaced by `CometProjectExec`.
diff --git a/contributor-guide/plugin_overview.html
b/contributor-guide/plugin_overview.html
index cfa5752099..ea966a2236 100644
--- a/contributor-guide/plugin_overview.html
+++ b/contributor-guide/plugin_overview.html
@@ -572,11 +572,14 @@ configuration.</p>
</section>
<section id="cometsparksessionextensions">
<h2>CometSparkSessionExtensions<a class="headerlink"
href="#cometsparksessionextensions" title="Link to this heading">#</a></h2>
-<p>On initialization, this class registers two physical plan optimization
rules with Spark: <code class="docutils literal notranslate"><span
class="pre">CometScanRule</span></code>
-and <code class="docutils literal notranslate"><span
class="pre">CometExecRule</span></code>. These rules run whenever a query stage
is being planned during Adaptive Query Execution, and
-run once for the entire plan when Adaptive Query Execution is disabled.</p>
-<section id="cometscanrule">
-<h3>CometScanRule<a class="headerlink" href="#cometscanrule" title="Link to
this heading">#</a></h3>
+<p>On initialization, this class registers one physical plan optimization rule
with Spark: <code class="docutils literal notranslate"><span
class="pre">CometRule</span></code>. It runs whenever
+a query stage is being planned during Adaptive Query Execution, and runs once
for the entire plan when Adaptive Query
+Execution is disabled.</p>
+<p><code class="docutils literal notranslate"><span
class="pre">CometRule</span></code> is two phases, applied in order: scan
conversion (<code class="docutils literal notranslate"><span
class="pre">CometScanRule</span></code>), then operator conversion
+(<code class="docutils literal notranslate"><span
class="pre">CometExecRule</span></code>). The order matters, because operator
conversion builds its native plan up from the nodes that scan
+conversion produces. Each phase is described below.</p>
+<section id="phase-1-cometscanrule">
+<h3>Phase 1: CometScanRule<a class="headerlink" href="#phase-1-cometscanrule"
title="Link to this heading">#</a></h3>
<p><code class="docutils literal notranslate"><span
class="pre">CometScanRule</span></code> replaces any Parquet scans with Comet
operators. There are different paths for Spark v1 and v2 data sources.</p>
<p>When reading from Parquet v1 data sources, Comet replaces <code
class="docutils literal notranslate"><span
class="pre">FileSourceScanExec</span></code> with a <code class="docutils
literal notranslate"><span class="pre">CometScanExec</span></code>, and for v2
data sources, <code class="docutils literal notranslate"><span
class="pre">BatchScanExec</span></code> is replaced with <code class="docutils
literal notranslate"><span class="pre">CometBatchScanExec</span></code>. In
both cases, Comet replaces Spark’s Parquet
@@ -589,8 +592,8 @@ convert the output from Spark’s scan to Arrow arrays. Note
that both <code cla
<p>Refer to the <a class="reference external"
href="https://datafusion.apache.org/comet/user-guide/datatypes.html";>Supported
Spark Data Types</a> section
in the contributor guide to see a list of currently supported data types.</p>
</section>
-<section id="cometexecrule">
-<h3>CometExecRule<a class="headerlink" href="#cometexecrule" title="Link to
this heading">#</a></h3>
+<section id="phase-2-cometexecrule">
+<h3>Phase 2: CometExecRule<a class="headerlink" href="#phase-2-cometexecrule"
title="Link to this heading">#</a></h3>
<p>This rule traverses bottom-up from the original Spark plan and attempts to
replace each operator with a Comet equivalent.
For example, a <code class="docutils literal notranslate"><span
class="pre">ProjectExec</span></code> will be replaced by <code class="docutils
literal notranslate"><span class="pre">CometProjectExec</span></code>.</p>
<p>When replacing a node, various checks are performed to determine if Comet
can support the operator and its expressions.
@@ -700,8 +703,8 @@ writing the batches to the shuffle file.</p>
<li class="toc-h1 nav-item toc-entry"><a class="reference internal nav-link"
href="#plugin-components">Plugin Components</a><ul class="visible nav
section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#comet-sql-plugin">Comet SQL Plugin</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#cometsparksessionextensions">CometSparkSessionExtensions</a><ul
class="nav section-nav flex-column">
-<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link"
href="#cometscanrule">CometScanRule</a></li>
-<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link"
href="#cometexecrule">CometExecRule</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link"
href="#phase-1-cometscanrule">Phase 1: CometScanRule</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link"
href="#phase-2-cometexecrule">Phase 2: CometExecRule</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#query-execution">Query Execution</a></li>
diff --git a/searchindex.js b/searchindex.js
index 38a2bd02d8..e3bf507992 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[56, "id1"]], "%": [[54, "id1"]], "&":
[[44, "id1"]], "*": [[54, "id2"]], "+": [[54, "id3"]], "-": [[54, "id4"]], "/":
[[54, "id5"]], "1. Format Your Code": [[41, "format-your-code"]], "1. Install
Comet": [[62, "install-comet"], [73, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[41, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[62,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[56, "id1"]], "%": [[54, "id1"]], "&":
[[44, "id1"]], "*": [[54, "id2"]], "+": [[54, "id3"]], "-": [[54, "id4"]], "/":
[[54, "id5"]], "1. Format Your Code": [[41, "format-your-code"]], "1. Install
Comet": [[62, "install-comet"], [73, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[41, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[62,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
