This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 5c37379a04 Publish built docs triggered by
9b608cc50922bfeac83dde2199847d1c509967cb
5c37379a04 is described below
commit 5c37379a0482b476de892e4a4babcd7480ba1730
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Sep 10 00:40:51 2026 +0000
Publish built docs triggered by 9b608cc50922bfeac83dde2199847d1c509967cb
---
.../contributor-guide/iceberg-spark-tests.md.txt | 22 ++++++++++++++++++++++
contributor-guide/iceberg-spark-tests.html | 19 +++++++++++++++++++
searchindex.js | 2 +-
3 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/_sources/contributor-guide/iceberg-spark-tests.md.txt
b/_sources/contributor-guide/iceberg-spark-tests.md.txt
index df6c168af6..4b0e827333 100644
--- a/_sources/contributor-guide/iceberg-spark-tests.md.txt
+++ b/_sources/contributor-guide/iceberg-spark-tests.md.txt
@@ -105,6 +105,28 @@ run against Spark 3.5.9 with Java 17; Iceberg 1.11.0 runs
against Spark 4.1.3 wi
(1.8, 1.9, 1.10) run only on pushes to main, or on a pull request labeled
`run-iceberg-tests`. All caller
workflows delegate to `iceberg_spark_test_reusable.yml`, which holds the build
and test job logic.
+The core Spark test target runs in four independent workers. The workflow
passes
+`dev/ci/iceberg-test-shards.gradle` as a Gradle init script: one worker runs
the long
+`TestStructuredStreamingRead` family, and the others hash the remaining class
names into three
+buckets. New tests are assigned automatically. Nested classes and all
parameterized cases stay
+with their enclosing class; Gradle's existing includes, exclusions, and JUnit
configuration are
+unchanged. The extensions and shaded-runtime targets remain unsharded.
+
+The matrix and partition count come from the same definition in
`dev/ci/check-iceberg-shards.py`;
+adding another matrix dimension does not change the partition count. Each
worker records its
+unsharded candidate set with only the Comet shard predicate disabled, then
restores the predicate
+before recording its selected set and executing tests. Both inventories and
the JUnit XML reports
+are uploaded. A dependent coverage job requires all shard indices, matching
unsharded inventories,
+and selected sets whose disjoint union equals that inventory. It downloads
only artifacts for the
+same Iceberg/Spark/Scala/JDK configuration in the current workflow run and
uses the latest available
+attempt per shard, so rerunning only failed jobs can reuse earlier successful
shards' inventories.
+
+These candidate inventories include classes that JUnit may not execute, so the
runtime job also
+runs `dev/ci/check-iceberg-shards.py`, a small Gradle/JUnit fixture that
checks the four shards'
+combined candidate classes and executed test cases equal an unsharded run
exactly once. It also
+checks nested, parameterized, inherited, and dynamically generated tests,
existing exclusions,
+and failure propagation. The fixture does not compile Spark or Iceberg.
+
Apply the `run-iceberg-tests` label to a pull request whenever it touches
reflection code
(`org.apache.comet.iceberg.IcebergReflection`) or other logic whose behavior
can differ across Iceberg
versions, since Iceberg 1.11 alone will not catch a regression that only
affects 1.8, 1.9, or 1.10.
diff --git a/contributor-guide/iceberg-spark-tests.html
b/contributor-guide/iceberg-spark-tests.html
index caa16d9242..5f27555609 100644
--- a/contributor-guide/iceberg-spark-tests.html
+++ b/contributor-guide/iceberg-spark-tests.html
@@ -634,6 +634,25 @@ run against Spark 3.5.9 with Java 17; Iceberg 1.11.0 runs
against Spark 4.1.3 wi
(the only version testing Spark 4.1) runs on every pull request and on pushes
to main; the older versions
(1.8, 1.9, 1.10) run only on pushes to main, or on a pull request labeled
<code class="docutils literal notranslate"><span
class="pre">run-iceberg-tests</span></code>. All caller
workflows delegate to <code class="docutils literal notranslate"><span
class="pre">iceberg_spark_test_reusable.yml</span></code>, which holds the
build and test job logic.</p>
+<p>The core Spark test target runs in four independent workers. The workflow
passes
+<code class="docutils literal notranslate"><span
class="pre">dev/ci/iceberg-test-shards.gradle</span></code> as a Gradle init
script: one worker runs the long
+<code class="docutils literal notranslate"><span
class="pre">TestStructuredStreamingRead</span></code> family, and the others
hash the remaining class names into three
+buckets. New tests are assigned automatically. Nested classes and all
parameterized cases stay
+with their enclosing class; Gradle’s existing includes, exclusions, and JUnit
configuration are
+unchanged. The extensions and shaded-runtime targets remain unsharded.</p>
+<p>The matrix and partition count come from the same definition in <code
class="docutils literal notranslate"><span
class="pre">dev/ci/check-iceberg-shards.py</span></code>;
+adding another matrix dimension does not change the partition count. Each
worker records its
+unsharded candidate set with only the Comet shard predicate disabled, then
restores the predicate
+before recording its selected set and executing tests. Both inventories and
the JUnit XML reports
+are uploaded. A dependent coverage job requires all shard indices, matching
unsharded inventories,
+and selected sets whose disjoint union equals that inventory. It downloads
only artifacts for the
+same Iceberg/Spark/Scala/JDK configuration in the current workflow run and
uses the latest available
+attempt per shard, so rerunning only failed jobs can reuse earlier successful
shards’ inventories.</p>
+<p>These candidate inventories include classes that JUnit may not execute, so
the runtime job also
+runs <code class="docutils literal notranslate"><span
class="pre">dev/ci/check-iceberg-shards.py</span></code>, a small Gradle/JUnit
fixture that checks the four shards’
+combined candidate classes and executed test cases equal an unsharded run
exactly once. It also
+checks nested, parameterized, inherited, and dynamically generated tests,
existing exclusions,
+and failure propagation. The fixture does not compile Spark or Iceberg.</p>
<p>Apply the <code class="docutils literal notranslate"><span
class="pre">run-iceberg-tests</span></code> label to a pull request whenever it
touches reflection code
(<code class="docutils literal notranslate"><span
class="pre">org.apache.comet.iceberg.IcebergReflection</span></code>) or other
logic whose behavior can differ across Iceberg
versions, since Iceberg 1.11 alone will not catch a regression that only
affects 1.8, 1.9, or 1.10.</p>
diff --git a/searchindex.js b/searchindex.js
index c7d9c091f6..6e807d2196 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[55, "id1"]], "%": [[53, "id1"]], "&":
[[43, "id1"]], "*": [[53, "id2"]], "+": [[53, "id3"]], "-": [[53, "id4"]], "/":
[[53, "id5"]], "1. Format Your Code": [[40, "format-your-code"]], "1. Install
Comet": [[61, "install-comet"], [71, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[40, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[61,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[55, "id1"]], "%": [[53, "id1"]], "&":
[[43, "id1"]], "*": [[53, "id2"]], "+": [[53, "id3"]], "-": [[53, "id4"]], "/":
[[53, "id5"]], "1. Format Your Code": [[40, "format-your-code"]], "1. Install
Comet": [[61, "install-comet"], [71, "install-comet"]], "1. Native Operators
(nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and
Verify": [[40, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[61,
"clone-iceberg-and-apply- [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
