This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9d0bf9865 Publish built docs triggered by
cdfdc2146f879bf9246361687ec4d0d25eb2a87e
9d0bf9865 is described below
commit 9d0bf9865e79229bce08357e6a00e519cb4b90c3
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Jun 6 19:00:02 2025 +0000
Publish built docs triggered by cdfdc2146f879bf9246361687ec4d0d25eb2a87e
---
_sources/user-guide/compatibility.md.txt | 9 +--
index.html | 1 -
searchindex.js | 2 +-
user-guide/compatibility.html | 101 +++++++++++++------------------
4 files changed, 47 insertions(+), 66 deletions(-)
diff --git a/_sources/user-guide/compatibility.md.txt
b/_sources/user-guide/compatibility.md.txt
index 212dc0dc3..39cd3a058 100644
--- a/_sources/user-guide/compatibility.md.txt
+++ b/_sources/user-guide/compatibility.md.txt
@@ -29,12 +29,6 @@ Comet aims to provide consistent results with the version of
Apache Spark that i
This guide offers information about areas of functionality where there are
known differences.
-# Compatibility Guide
-
-Comet aims to provide consistent results with the version of Apache Spark that
is being used.
-
-This guide offers information about areas of functionality where there are
known differences.
-
## Parquet Scans
Comet currently has three distinct implementations of the Parquet scan
operator. The configuration property
@@ -89,6 +83,9 @@ because they are handled well in Spark (e.g.,
`SQLOrderingUtil.compareFloats`).
functions of arrow-rs used by DataFusion do not normalize NaN and zero (e.g.,
[arrow::compute::kernels::cmp::eq](https://docs.rs/arrow/latest/arrow/compute/kernels/cmp/fn.eq.html#)).
So Comet will add additional normalization expression of NaN and zero for
comparison.
+There is a known bug with using count(distinct) within aggregate queries,
where each NaN value will be counted
+separately [#1824](https://github.com/apache/datafusion-comet/issues/1824).
+
## Incompatible Expressions
Some Comet native expressions are not 100% compatible with Spark and are
disabled by default. These expressions
diff --git a/index.html b/index.html
index af2348b85..b7a5d04b6 100644
--- a/index.html
+++ b/index.html
@@ -336,7 +336,6 @@ as a native runtime to achieve improvement in terms of
query efficiency and quer
<li class="toctree-l1"><a class="reference internal"
href="user-guide/expressions.html">Supported Expressions</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/configs.html">Configuration Settings</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/compatibility.html">Compatibility Guide</a></li>
-<li class="toctree-l1"><a class="reference internal"
href="user-guide/compatibility.html#id1">Compatibility Guide</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/tuning.html">Tuning Guide</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/metrics.html">Metrics Guide</a></li>
</ul>
diff --git a/searchindex.js b/searchindex.js
index 567e9c355..ec816c184 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]],
"2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[14,
"ansi-mode"]], "API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[13, null]],
"Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[19,
null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]],
"2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run
Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[14,
"ansi-mode"]], "API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[13, null]],
"Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[19,
null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
diff --git a/user-guide/compatibility.html b/user-guide/compatibility.html
index 86bd22dff..a09f16ae5 100644
--- a/user-guide/compatibility.html
+++ b/user-guide/compatibility.html
@@ -291,67 +291,55 @@ under the License.
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
- <li class="toc-h1 nav-item toc-entry">
- <a class="reference internal nav-link" href="#">
- Compatibility Guide
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#parquet-scans">
+ Parquet Scans
</a>
</li>
- <li class="toc-h1 nav-item toc-entry">
- <a class="reference internal nav-link" href="#id1">
- Compatibility Guide
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#ansi-mode">
+ ANSI mode
</a>
- <ul class="visible nav section-nav flex-column">
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#parquet-scans">
- Parquet Scans
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#ansi-mode">
- ANSI mode
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#floating-number-comparison">
- Floating number comparison
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#incompatible-expressions">
- Incompatible Expressions
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#array-expressions">
- Array Expressions
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#floating-number-comparison">
+ Floating number comparison
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#incompatible-expressions">
+ Incompatible Expressions
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#array-expressions">
+ Array Expressions
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#regular-expressions">
+ Regular Expressions
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#cast">
+ Cast
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#compatible-casts">
+ Compatible Casts
</a>
</li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#regular-expressions">
- Regular Expressions
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#incompatible-casts">
+ Incompatible Casts
</a>
</li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#cast">
- Cast
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#unsupported-casts">
+ Unsupported Casts
</a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#compatible-casts">
- Compatible Casts
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#incompatible-casts">
- Incompatible Casts
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#unsupported-casts">
- Unsupported Casts
- </a>
- </li>
- </ul>
</li>
</ul>
</li>
@@ -410,11 +398,6 @@ TO MODIFY THIS CONTENT MAKE SURE THAT YOU MAKE YOUR
CHANGES TO THE TEMPLATE FILE
<h1>Compatibility Guide<a class="headerlink" href="#compatibility-guide"
title="Link to this heading">¶</a></h1>
<p>Comet aims to provide consistent results with the version of Apache Spark
that is being used.</p>
<p>This guide offers information about areas of functionality where there are
known differences.</p>
-</section>
-<section id="id1">
-<h1>Compatibility Guide<a class="headerlink" href="#id1" title="Link to this
heading">¶</a></h1>
-<p>Comet aims to provide consistent results with the version of Apache Spark
that is being used.</p>
-<p>This guide offers information about areas of functionality where there are
known differences.</p>
<section id="parquet-scans">
<h2>Parquet Scans<a class="headerlink" href="#parquet-scans" title="Link to
this heading">¶</a></h2>
<p>Comet currently has three distinct implementations of the Parquet scan
operator. The configuration property
@@ -475,6 +458,8 @@ However, one exception is comparison. Spark does not
normalize NaN and zero when
because they are handled well in Spark (e.g., <code class="docutils literal
notranslate"><span class="pre">SQLOrderingUtil.compareFloats</span></code>).
But the comparison
functions of arrow-rs used by DataFusion do not normalize NaN and zero (e.g.,
<a class="reference external"
href="https://docs.rs/arrow/latest/arrow/compute/kernels/cmp/fn.eq.html#";>arrow::compute::kernels::cmp::eq</a>).
So Comet will add additional normalization expression of NaN and zero for
comparison.</p>
+<p>There is a known bug with using count(distinct) within aggregate queries,
where each NaN value will be counted
+separately <a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/1824";>#1824</a>.</p>
</section>
<section id="incompatible-expressions">
<h2>Incompatible Expressions<a class="headerlink"
href="#incompatible-expressions" title="Link to this heading">¶</a></h2>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@datafusion.apache.org
For additional commands, e-mail: commits-h...@datafusion.apache.org
