This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1d2d4e8bf6 Publish built docs triggered by
2e907e5602425b7d671ba185b5cd0a6b7b68a490
1d2d4e8bf6 is described below
commit 1d2d4e8bf6f8676fa0f46e5ba3877006a7f0af47
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Jul 24 18:44:08 2026 +0000
Publish built docs triggered by 2e907e5602425b7d671ba185b5cd0a6b7b68a490
---
.../expression-audits/agg_funcs.md.txt | 9 ++++++
_sources/user-guide/latest/configs.md.txt | 1 +
_sources/user-guide/latest/expressions.md.txt | 4 +--
contributor-guide/expression-audits/agg_funcs.html | 12 ++++++++
searchindex.js | 2 +-
user-guide/latest/configs.html | 36 ++++++++++++----------
user-guide/latest/expressions.html | 8 ++---
7 files changed, 49 insertions(+), 23 deletions(-)
diff --git a/_sources/contributor-guide/expression-audits/agg_funcs.md.txt
b/_sources/contributor-guide/expression-audits/agg_funcs.md.txt
index 3904cca30c..63f4ca84b7 100644
--- a/_sources/contributor-guide/expression-audits/agg_funcs.md.txt
+++ b/_sources/contributor-guide/expression-audits/agg_funcs.md.txt
@@ -51,6 +51,15 @@
- Spark 3.5.8 (2026-05-26)
- Spark 4.0.1 (2026-05-26)
+## collect_list
+
+- Spark 3.4.3 (audited 2026-06-24): `CollectList` extends
`Collect[ArrayBuffer[Any]]`, returns `ArrayType(child.dataType, containsNull =
false)`, ignores NULL inputs in `update()` (Hive-compatible semantics), and
yields an empty array as `defaultResult`. `nullable = false`. No
`checkInputDataTypes` override, so any input type is accepted (including
STRUCT, ARRAY, MAP). Registered as both `collect_list` and `array_agg` aliases
in `FunctionRegistry`.
+- Spark 3.5.8 (audited 2026-06-24): identical to 3.4.3.
+- Spark 4.0.1 (audited 2026-06-24): only structural change is adding `with
UnaryLike[Expression]` to the case class (no behavior change).
+- Spark 4.1.1 (audited 2026-06-24): identical to 4.0.1.
+- Comet implementation: native side delegates to
`datafusion_spark::function::aggregate::collect::SparkCollectList`, which wraps
`ArrayAggAccumulator` with `ignore_nulls = true` and converts a final NULL
accumulator state to an empty array (matching Spark's `defaultResult`). The
native return type is `List(Field, containsNull = true)`, while Spark uses
`containsNull = false`. Because nulls are filtered before insertion, no nulls
actually appear in the array, so this is a schema-shape dif [...]
+- Spark 4.2 (preview): `CollectList` and `CollectSet` gain an `ignoreNulls`
field (default `true`); `RESPECT NULLS` sets it to `false` and keeps null
elements. The native path always drops nulls, so `CometCollectShim` reads the
field per Spark version (always `true` on 3.4-4.1) and `CometCollectList` /
`CometCollectSet` report `Unsupported` when it is `false`, falling back to
Spark.
+
## median
- Spark 3.4.3 (audited 2026-06-24): `Median(child)` is a
`RuntimeReplaceableAggregate` with `replacement = Percentile(child,
Literal(0.5))`. Catalyst rewrites `median(x)` to `percentile(x, 0.5)` before
Comet sees the plan, so it is served by `CometPercentile`.
diff --git a/_sources/user-guide/latest/configs.md.txt
b/_sources/user-guide/latest/configs.md.txt
index a84b726a81..8926f2a8ee 100644
--- a/_sources/user-guide/latest/configs.md.txt
+++ b/_sources/user-guide/latest/configs.md.txt
@@ -467,6 +467,7 @@ These settings can be used to determine which parts of the
plan are accelerated
| `spark.comet.expression.BitOrAgg.enabled` | Enable Comet acceleration for
`BitOrAgg` | true |
| `spark.comet.expression.BitXorAgg.enabled` | Enable Comet acceleration for
`BitXorAgg` | true |
| `spark.comet.expression.BloomFilterAggregate.enabled` | Enable Comet
acceleration for `BloomFilterAggregate` | true |
+| `spark.comet.expression.CollectList.enabled` | Enable Comet acceleration for
`CollectList` | true |
| `spark.comet.expression.CollectSet.enabled` | Enable Comet acceleration for
`CollectSet` | true |
| `spark.comet.expression.Corr.enabled` | Enable Comet acceleration for `Corr`
| true |
| `spark.comet.expression.Count.enabled` | Enable Comet acceleration for
`Count` | true |
diff --git a/_sources/user-guide/latest/expressions.md.txt
b/_sources/user-guide/latest/expressions.md.txt
index 6323083cbc..efbd33ba1c 100644
--- a/_sources/user-guide/latest/expressions.md.txt
+++ b/_sources/user-guide/latest/expressions.md.txt
@@ -72,14 +72,14 @@ The tables below list every Spark built-in expression with
its current status.
| `any_value` | ✅ | |
| `approx_count_distinct` | ✅ | |
| `approx_percentile` | ✅ | Byte, short, int, long, float, and double input;
other input types fall back to Spark |
-| `array_agg` | 🔜 | Array aggregate (related to `collect_list`,
[#2524](https://github.com/apache/datafusion-comet/issues/2524)) |
+| `array_agg` | ✅ | Alias for `collect_list` |
| `avg` | ✅ | Interval types fall back |
| `bit_and` | ✅ | |
| `bit_or` | ✅ | |
| `bit_xor` | ✅ | |
| `bool_and` | ✅ | |
| `bool_or` | ✅ | |
-| `collect_list` | 🔜 |
[#2524](https://github.com/apache/datafusion-comet/issues/2524) |
+| `collect_list` | ✅ | |
| `collect_set` | ✅ | |
| `corr` | ✅ | |
| `count` | ✅ | |
diff --git a/contributor-guide/expression-audits/agg_funcs.html
b/contributor-guide/expression-audits/agg_funcs.html
index c26cd11d28..006cce6117 100644
--- a/contributor-guide/expression-audits/agg_funcs.html
+++ b/contributor-guide/expression-audits/agg_funcs.html
@@ -592,6 +592,17 @@ under the License.
<li><p>Spark 4.0.1 (2026-05-26)</p></li>
</ul>
</section>
+<section id="collect-list">
+<h2>collect_list<a class="headerlink" href="#collect-list" title="Link to this
heading">#</a></h2>
+<ul class="simple">
+<li><p>Spark 3.4.3 (audited 2026-06-24): <code class="docutils literal
notranslate"><span class="pre">CollectList</span></code> extends <code
class="docutils literal notranslate"><span
class="pre">Collect[ArrayBuffer[Any]]</span></code>, returns <code
class="docutils literal notranslate"><span
class="pre">ArrayType(child.dataType,</span> <span
class="pre">containsNull</span> <span class="pre">=</span> <span
class="pre">false)</span></code>, ignores NULL inputs in <code class="docutils
li [...]
+<li><p>Spark 3.5.8 (audited 2026-06-24): identical to 3.4.3.</p></li>
+<li><p>Spark 4.0.1 (audited 2026-06-24): only structural change is adding
<code class="docutils literal notranslate"><span class="pre">with</span> <span
class="pre">UnaryLike[Expression]</span></code> to the case class (no behavior
change).</p></li>
+<li><p>Spark 4.1.1 (audited 2026-06-24): identical to 4.0.1.</p></li>
+<li><p>Comet implementation: native side delegates to <code class="docutils
literal notranslate"><span
class="pre">datafusion_spark::function::aggregate::collect::SparkCollectList</span></code>,
which wraps <code class="docutils literal notranslate"><span
class="pre">ArrayAggAccumulator</span></code> with <code class="docutils
literal notranslate"><span class="pre">ignore_nulls</span> <span
class="pre">=</span> <span class="pre">true</span></code> and converts a final
NULL accumulator st [...]
+<li><p>Spark 4.2 (preview): <code class="docutils literal notranslate"><span
class="pre">CollectList</span></code> and <code class="docutils literal
notranslate"><span class="pre">CollectSet</span></code> gain an <code
class="docutils literal notranslate"><span
class="pre">ignoreNulls</span></code> field (default <code class="docutils
literal notranslate"><span class="pre">true</span></code>); <code
class="docutils literal notranslate"><span class="pre">RESPECT</span> <span
class="pre">N [...]
+</ul>
+</section>
<section id="median">
<h2>median<a class="headerlink" href="#median" title="Link to this
heading">#</a></h2>
<ul class="simple">
@@ -665,6 +676,7 @@ under the License.
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#approx-percentile">approx_percentile</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#avg">avg</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#bit-and">bit_and</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#collect-list">collect_list</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#median">median</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#percentile">percentile</a></li>
</ul>
diff --git a/searchindex.js b/searchindex.js
index 355af01c27..1758e6d69d 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[53, "id1"]], "%": [[51, "id1"]], "&":
[[41, "id1"]], "*": [[51, "id2"]], "+": [[51, "id3"]], "-": [[51, "id4"]], "/":
[[51, "id5"]], "1. Format Your Code": [[38, "format-your-code"]], "1. Install
Comet": [[59, "install-comet"], [69, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[26, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[38, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[59,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[53, "id1"]], "%": [[51, "id1"]], "&":
[[41, "id1"]], "*": [[51, "id2"]], "+": [[51, "id3"]], "-": [[51, "id4"]], "/":
[[51, "id5"]], "1. Format Your Code": [[38, "format-your-code"]], "1. Install
Comet": [[59, "install-comet"], [69, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[26, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[38, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[59,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/configs.html b/user-guide/latest/configs.html
index 4d4c362cdf..99479ac007 100644
--- a/user-guide/latest/configs.html
+++ b/user-guide/latest/configs.html
@@ -2345,67 +2345,71 @@ under the License.
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">BloomFilterAggregate</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CollectSet.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CollectList.enabled</span></code></p></td>
+<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">CollectList</span></code></p></td>
+<td><p>true</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CollectSet.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">CollectSet</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Corr.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Corr.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Corr</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Count.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Count.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Count</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CovPopulation.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CovPopulation.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">CovPopulation</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CovSample.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.CovSample.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">CovSample</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.First.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.First.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">First</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.HyperLogLogPlusPlus.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.HyperLogLogPlusPlus.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">HyperLogLogPlusPlus</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Last.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Last.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Last</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Max.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Max.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Max</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Min.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Min.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Min</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Percentile.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Percentile.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Percentile</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.StddevPop.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.StddevPop.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">StddevPop</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.StddevSamp.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.StddevSamp.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">StddevSamp</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Sum.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.Sum.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">Sum</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.VariancePop.enabled</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.VariancePop.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">VariancePop</span></code></p></td>
<td><p>true</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.VarianceSamp.enabled</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">spark.comet.expression.VarianceSamp.enabled</span></code></p></td>
<td><p>Enable Comet acceleration for <code class="docutils literal
notranslate"><span class="pre">VarianceSamp</span></code></p></td>
<td><p>true</p></td>
</tr>
diff --git a/user-guide/latest/expressions.html
b/user-guide/latest/expressions.html
index 72c124d75e..930424337e 100644
--- a/user-guide/latest/expressions.html
+++ b/user-guide/latest/expressions.html
@@ -819,8 +819,8 @@ expressions. The following function families are
<strong>not currently planned</
<td><p>Byte, short, int, long, float, and double input; other input types fall
back to Spark</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">array_agg</span></code></p></td>
-<td><p>🔜</p></td>
-<td><p>Array aggregate (related to <code class="docutils literal
notranslate"><span class="pre">collect_list</span></code>, <a class="reference
external"
href="https://github.com/apache/datafusion-comet/issues/2524";>#2524</a>)</p></td>
+<td><p>✅</p></td>
+<td><p>Alias for <code class="docutils literal notranslate"><span
class="pre">collect_list</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">avg</span></code></p></td>
<td><p>✅</p></td>
@@ -847,8 +847,8 @@ expressions. The following function families are
<strong>not currently planned</
<td><p></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">collect_list</span></code></p></td>
-<td><p>🔜</p></td>
-<td><p><a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/2524";>#2524</a></p></td>
+<td><p>✅</p></td>
+<td><p></p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">collect_set</span></code></p></td>
<td><p>✅</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
