This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 843dc360f8 Publish built docs triggered by
bc53ad5c56eff6e4e3c2d702a6869b7cf28352ae
843dc360f8 is described below
commit 843dc360f8fa4a6fa7b22a9234e9b006afb34bad
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Jul 23 22:34:25 2026 +0000
Publish built docs triggered by bc53ad5c56eff6e4e3c2d702a6869b7cf28352ae
---
_sources/contributor-guide/roadmap.md.txt | 6 +++---
_sources/user-guide/latest/iceberg.md.txt | 22 ++++++++++++++++------
contributor-guide/roadmap.html | 4 ++--
searchindex.js | 2 +-
user-guide/latest/iceberg.html | 22 ++++++++++++++++------
5 files changed, 38 insertions(+), 18 deletions(-)
diff --git a/_sources/contributor-guide/roadmap.md.txt
b/_sources/contributor-guide/roadmap.md.txt
index b8ae8a310f..59aba8e7c4 100644
--- a/_sources/contributor-guide/roadmap.md.txt
+++ b/_sources/contributor-guide/roadmap.md.txt
@@ -52,14 +52,14 @@ significant family of Spark expressions in one effort.
Native Parquet scans (`CometNativeScanExec`) support Dynamic Partition Pruning
(DPP) both with and without
Adaptive Query Execution. Non-AQE DPP landed in [#4011] and AQE DPP with
broadcast reuse landed in [#4112].
-Iceberg native scans currently support non-AQE DPP only ([#3349], [#3511]);
extending broadcast reuse to AQE
-DPP for Iceberg is tracked at [#3510].
+Iceberg native scans support non-AQE DPP ([#3349], [#3511]) and, on Spark
3.5+, AQE DPP with broadcast reuse
+([#4215]); on Spark 3.4 Iceberg AQE DPP falls back to Spark without reuse.
[#3349]: https://github.com/apache/datafusion-comet/pull/3349
-[#3510]: https://github.com/apache/datafusion-comet/issues/3510
[#3511]: https://github.com/apache/datafusion-comet/pull/3511
[#4011]: https://github.com/apache/datafusion-comet/pull/4011
[#4112]: https://github.com/apache/datafusion-comet/pull/4112
+[#4215]: https://github.com/apache/datafusion-comet/pull/4215
## TPC-H and TPC-DS Performance
diff --git a/_sources/user-guide/latest/iceberg.md.txt
b/_sources/user-guide/latest/iceberg.md.txt
index b0cd7eeed3..13009f3c47 100644
--- a/_sources/user-guide/latest/iceberg.md.txt
+++ b/_sources/user-guide/latest/iceberg.md.txt
@@ -26,7 +26,7 @@ then serialized to Comet's native execution engine (see
[PR #2528](https://github.com/apache/datafusion-comet/pull/2528)).
The example below uses Spark's package downloader to retrieve Comet
1.0.0-SNAPSHOT and Iceberg
-1.8.1, but Comet has been tested with Iceberg 1.5, 1.7, 1.8, 1.9, and 1.10.
The native Iceberg
+1.8.1, but Comet has been tested with Iceberg 1.5, 1.7, 1.8, 1.9, 1.10, and
1.11. The native Iceberg
reader is enabled by default. To disable it, set
`spark.comet.scan.icebergNative.enabled=false`.
The example uses the Spark 3.5 / Scala 2.12 build of Comet; substitute the
Comet artifact
@@ -63,7 +63,15 @@ The native Iceberg reader supports the following features:
**Table specifications:**
-- Iceberg table spec v1 and v2 (v3 will fall back to Spark)
+- Iceberg table spec v1, v2, and v3
+
+**Encryption:**
+
+- Encrypted v3 tables using 128-bit or 256-bit AES-GCM data keys (requires
Iceberg 1.11 or newer).
+ Iceberg-Java unwraps the key envelope on the driver during planning and
stores the plaintext data
+ key in each file's `key_metadata`, which the native reader uses directly, so
no KMS integration is
+ needed on the native side. Iceberg-Java also permits 192-bit data keys;
those tables fall back to
+ Spark (no AES-192-GCM in the underlying crypto).
**Schema and data types:**
@@ -156,17 +164,19 @@ These `s3.*` storage properties are not specific to the
Hive catalog shown here.
### Current limitations
-The following scenarios will fall back to Spark's native Iceberg reader:
+The following scenarios will fall back to the JVM Iceberg reader:
-- Iceberg table spec v3 scans
+- Iceberg table spec v4 or newer
+- v3 tables with columns that declare an initial default value
+- v3 column types the native reader cannot read (`variant`, `geometry`,
`geography`, `unknown`)
+- Encrypted tables with 192-bit data keys (no AES-192-GCM in the underlying
crypto)
+- Deletion vectors (v3 Puffin deletes); positional and equality deletes in
Parquet are supported
- Iceberg writes (reads are accelerated, writes use Spark)
- Tables backed by Avro or ORC data files (only Parquet is accelerated)
- Tables partitioned on `BINARY` or `DECIMAL` (with precision >28) columns
- Scans with residual filters using `truncate`, `bucket`, `year`, `month`,
`day`, or `hour`
transform functions (partition pruning still works, but row-level filtering
of these
transforms falls back)
-- Dynamic Partition Pruning under Adaptive Query Execution (non-AQE DPP is
supported);
- see [#3510](https://github.com/apache/datafusion-comet/issues/3510)
### Iceberg UDFs
diff --git a/contributor-guide/roadmap.html b/contributor-guide/roadmap.html
index 1931e5aa9c..348c2a352d 100644
--- a/contributor-guide/roadmap.html
+++ b/contributor-guide/roadmap.html
@@ -571,8 +571,8 @@ significant family of Spark expressions in one effort.</p>
<h2>Dynamic Partition Pruning<a class="headerlink"
href="#dynamic-partition-pruning" title="Link to this heading">#</a></h2>
<p>Native Parquet scans (<code class="docutils literal notranslate"><span
class="pre">CometNativeScanExec</span></code>) support Dynamic Partition
Pruning (DPP) both with and without
Adaptive Query Execution. Non-AQE DPP landed in <a class="reference external"
href="https://github.com/apache/datafusion-comet/pull/4011";>#4011</a> and AQE
DPP with broadcast reuse landed in <a class="reference external"
href="https://github.com/apache/datafusion-comet/pull/4112";>#4112</a>.
-Iceberg native scans currently support non-AQE DPP only (<a class="reference
external"
href="https://github.com/apache/datafusion-comet/pull/3349";>#3349</a>, <a
class="reference external"
href="https://github.com/apache/datafusion-comet/pull/3511";>#3511</a>);
extending broadcast reuse to AQE
-DPP for Iceberg is tracked at <a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/3510";>#3510</a>.</p>
+Iceberg native scans support non-AQE DPP (<a class="reference external"
href="https://github.com/apache/datafusion-comet/pull/3349";>#3349</a>, <a
class="reference external"
href="https://github.com/apache/datafusion-comet/pull/3511";>#3511</a>) and, on
Spark 3.5+, AQE DPP with broadcast reuse
+(<a class="reference external"
href="https://github.com/apache/datafusion-comet/pull/4215";>#4215</a>); on
Spark 3.4 Iceberg AQE DPP falls back to Spark without reuse.</p>
</section>
<section id="tpc-h-and-tpc-ds-performance">
<h2>TPC-H and TPC-DS Performance<a class="headerlink"
href="#tpc-h-and-tpc-ds-performance" title="Link to this heading">#</a></h2>
diff --git a/searchindex.js b/searchindex.js
index 50aeda3cd2..89647a57d1 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[53, "id1"]], "%": [[51, "id1"]], "&":
[[41, "id1"]], "*": [[51, "id2"]], "+": [[51, "id3"]], "-": [[51, "id4"]], "/":
[[51, "id5"]], "1. Format Your Code": [[38, "format-your-code"]], "1. Install
Comet": [[59, "install-comet"], [69, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[26, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[38, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[59,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[53, "id1"]], "%": [[51, "id1"]], "&":
[[41, "id1"]], "*": [[51, "id2"]], "+": [[51, "id3"]], "-": [[51, "id4"]], "/":
[[51, "id5"]], "1. Format Your Code": [[38, "format-your-code"]], "1. Install
Comet": [[59, "install-comet"], [69, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[26, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[38, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[59,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/iceberg.html b/user-guide/latest/iceberg.html
index facad01623..05a56dc38c 100644
--- a/user-guide/latest/iceberg.html
+++ b/user-guide/latest/iceberg.html
@@ -747,7 +747,7 @@ under the License.
then serialized to Comet’s native execution engine (see
<a class="reference external"
href="https://github.com/apache/datafusion-comet/pull/2528";>PR #2528</a>).</p>
<p>The example below uses Spark’s package downloader to retrieve Comet
1.0.0-SNAPSHOT and Iceberg
-1.8.1, but Comet has been tested with Iceberg 1.5, 1.7, 1.8, 1.9, and 1.10.
The native Iceberg
+1.8.1, but Comet has been tested with Iceberg 1.5, 1.7, 1.8, 1.9, 1.10, and
1.11. The native Iceberg
reader is enabled by default. To disable it, set <code class="docutils literal
notranslate"><span
class="pre">spark.comet.scan.icebergNative.enabled=false</span></code>.</p>
<p>The example uses the Spark 3.5 / Scala 2.12 build of Comet; substitute the
Comet artifact
matching your Spark and Scala versions (Comet also ships Spark 3.5 / Scala
2.13 and Spark
@@ -779,7 +779,15 @@ values between 2 and 8 based on your workload.</p>
<p>The native Iceberg reader supports the following features:</p>
<p><strong>Table specifications:</strong></p>
<ul class="simple">
-<li><p>Iceberg table spec v1 and v2 (v3 will fall back to Spark)</p></li>
+<li><p>Iceberg table spec v1, v2, and v3</p></li>
+</ul>
+<p><strong>Encryption:</strong></p>
+<ul class="simple">
+<li><p>Encrypted v3 tables using 128-bit or 256-bit AES-GCM data keys
(requires Iceberg 1.11 or newer).
+Iceberg-Java unwraps the key envelope on the driver during planning and stores
the plaintext data
+key in each file’s <code class="docutils literal notranslate"><span
class="pre">key_metadata</span></code>, which the native reader uses directly,
so no KMS integration is
+needed on the native side. Iceberg-Java also permits 192-bit data keys; those
tables fall back to
+Spark (no AES-192-GCM in the underlying crypto).</p></li>
</ul>
<p><strong>Schema and data types:</strong></p>
<ul class="simple">
@@ -867,17 +875,19 @@ configure Spark to use a REST catalog with Comet’s native
Iceberg scan:</p>
</section>
<section id="current-limitations">
<h3>Current limitations<a class="headerlink" href="#current-limitations"
title="Link to this heading">#</a></h3>
-<p>The following scenarios will fall back to Spark’s native Iceberg reader:</p>
+<p>The following scenarios will fall back to the JVM Iceberg reader:</p>
<ul class="simple">
-<li><p>Iceberg table spec v3 scans</p></li>
+<li><p>Iceberg table spec v4 or newer</p></li>
+<li><p>v3 tables with columns that declare an initial default value</p></li>
+<li><p>v3 column types the native reader cannot read (<code class="docutils
literal notranslate"><span class="pre">variant</span></code>, <code
class="docutils literal notranslate"><span class="pre">geometry</span></code>,
<code class="docutils literal notranslate"><span
class="pre">geography</span></code>, <code class="docutils literal
notranslate"><span class="pre">unknown</span></code>)</p></li>
+<li><p>Encrypted tables with 192-bit data keys (no AES-192-GCM in the
underlying crypto)</p></li>
+<li><p>Deletion vectors (v3 Puffin deletes); positional and equality deletes
in Parquet are supported</p></li>
<li><p>Iceberg writes (reads are accelerated, writes use Spark)</p></li>
<li><p>Tables backed by Avro or ORC data files (only Parquet is
accelerated)</p></li>
<li><p>Tables partitioned on <code class="docutils literal notranslate"><span
class="pre">BINARY</span></code> or <code class="docutils literal
notranslate"><span class="pre">DECIMAL</span></code> (with precision >28)
columns</p></li>
<li><p>Scans with residual filters using <code class="docutils literal
notranslate"><span class="pre">truncate</span></code>, <code class="docutils
literal notranslate"><span class="pre">bucket</span></code>, <code
class="docutils literal notranslate"><span class="pre">year</span></code>,
<code class="docutils literal notranslate"><span
class="pre">month</span></code>, <code class="docutils literal
notranslate"><span class="pre">day</span></code>, or <code class="docutils
literal notrans [...]
transform functions (partition pruning still works, but row-level filtering of
these
transforms falls back)</p></li>
-<li><p>Dynamic Partition Pruning under Adaptive Query Execution (non-AQE DPP
is supported);
-see <a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/3510";>#3510</a></p></li>
</ul>
</section>
<section id="iceberg-udfs">
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
