This is an automated email from the ASF dual-hosted git repository.
andygrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new af534e0fa0 docs: document ReusedExchange caveat and both
CometSparkToColumnar names in operator-count exclusions (#5240)
af534e0fa0 is described below
commit af534e0fa01a2d9db47aa5aa43fc56f0c5908a72
Author: Andy Grove <[email protected]>
AuthorDate: Mon Aug 3 16:14:34 2026 -0600
docs: document ReusedExchange caveat and both CometSparkToColumnar names in
operator-count exclusions (#5240)
Follow-up to review feedback on #5206.
- The transition-nodes bullet only listed `CometSparkRowToColumnar`, but
`CometSparkToColumnarExec` also renders as `CometSparkColumnarToColumnar`
when its child supports columnar. Both names are excluded from the
operator
counts today (the match is on the Scala type, not the rendered name), so
list
both.
- `ReusedExchangeExec` is matched in the same ignore arm as the other
wrapper
nodes but was not mentioned. It also behaves differently from them:
`getActualPlan` unwraps it to its child, so a reused subtree is counted
once
per reference rather than once per plan. Document it along with that
caveat
and the link to item 3 of #5203.
---
docs/source/user-guide/latest/understanding-comet-plans.md | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/docs/source/user-guide/latest/understanding-comet-plans.md
b/docs/source/user-guide/latest/understanding-comet-plans.md
index a05d271360..ea32e37b8f 100644
--- a/docs/source/user-guide/latest/understanding-comet-plans.md
+++ b/docs/source/user-guide/latest/understanding-comet-plans.md
@@ -182,12 +182,22 @@ Not every node in the plan is an eligible operator. The
following are excluded
from both operator counts:
- Transition nodes (`CometColumnarToRow`, `CometNativeColumnarToRow`,
- `CometSparkRowToColumnar`, `ColumnarToRow`, `RowToColumnar`), which are
- reported separately as the transition count.
+ `CometSparkRowToColumnar`, `CometSparkColumnarToColumnar`, `ColumnarToRow`,
+ `RowToColumnar`), which are reported separately as the transition count.
+ `CometSparkRowToColumnar` and `CometSparkColumnarToColumnar` are the two
names
+ a single operator renders under, depending on whether its child already
+ produces columnar data, and both are excluded.
- Wrappers that do no work of their own: `AdaptiveSparkPlan`, `InputAdapter`,
`WholeStageCodegen`, query stages, and `AQEShuffleRead`.
- The reuse marker `ReusedSubquery`. The subquery it points at is counted where
that subquery is shown, so the marker itself does not add to the totals.
+- `ReusedExchange`, but with a caveat: it is not cleanly excluded the way the
+ wrappers above are. The node itself is skipped, and yet walking the plan
+ replaces it with the exchange it reuses, so the reused subtree is counted
once
+ per reference rather than once for the whole plan. A plan that reuses one
+ exchange in three places contributes that subtree's operators three times.
+ Counting reused exchanges once is tracked as item 3 of
+ [#5203](https://github.com/apache/datafusion-comet/issues/5203).
### `spark.comet.explain.native.enabled`
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
