This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new bfdc3258 Publish built docs triggered by
b3fc655ff2f87bcc342e50cfe4e71842ee108e6e
bfdc3258 is described below
commit bfdc325881aa902f52c9932c8c5de884c9a9f122
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Mon Aug 26 19:37:57 2024 +0000
Publish built docs triggered by b3fc655ff2f87bcc342e50cfe4e71842ee108e6e
---
_sources/user-guide/installation.md.txt | 12 +++++++-----
searchindex.js | 2 +-
user-guide/installation.html | 10 ++++++----
3 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/_sources/user-guide/installation.md.txt
b/_sources/user-guide/installation.md.txt
index 1efa11aa..c5ca9140 100644
--- a/_sources/user-guide/installation.md.txt
+++ b/_sources/user-guide/installation.md.txt
@@ -28,7 +28,7 @@ Make sure the following requirements are met and software
installed on your mach
## Requirements
-- Apache Spark 3.3, or 3.4
+- Apache Spark 3.3, 3.4, or 3.5
- JDK 8 and up
- GLIBC 2.17 (Centos 7) and up
@@ -79,7 +79,7 @@ make release-nogit PROFILES="-Pspark-3.4"
Make sure `SPARK_HOME` points to the same Spark version as Comet was built for.
```console
-export COMET_JAR=spark/target/comet-spark-spark3.4_2.12-0.1.0-SNAPSHOT.jar
+export COMET_JAR=spark/target/comet-spark-spark3.4_2.12-0.2.0-SNAPSHOT.jar
$SPARK_HOME/bin/spark-shell \
--jars $COMET_JAR \
@@ -88,7 +88,9 @@ $SPARK_HOME/bin/spark-shell \
--conf spark.plugins=org.apache.spark.CometPlugin \
--conf spark.comet.enabled=true \
--conf spark.comet.exec.enabled=true \
- --conf spark.comet.explainFallback.enabled=true
+ --conf spark.comet.explainFallback.enabled=true \
+ --conf spark.driver.memory=1g \
+ --conf spark.executor.memory=1g
```
### Verify Comet enabled for Spark SQL query
@@ -145,7 +147,7 @@ explicitly contain Comet otherwise Spark may use a
different class-loader for th
components which will then fail at runtime. For example:
```
---driver-class-path spark/target/comet-spark-spark3.4_2.12-0.1.0-SNAPSHOT.jar
+--driver-class-path spark/target/comet-spark-spark3.4_2.12-0.2.0-SNAPSHOT.jar
```
Some cluster managers may require additional configuration, see
<https://spark.apache.org/docs/latest/cluster-overview.html>
@@ -157,4 +159,4 @@ To enable columnar shuffle which supports all partitioning
and basic complex typ
```
### Memory tuning
-In addition to Apache Spark memory configuration parameters the Comet
introduces own parameters to configure memory allocation for native execution.
More [Comet Memory Tuning](./tuning.md)
\ No newline at end of file
+In addition to Apache Spark memory configuration parameters the Comet
introduces own parameters to configure memory allocation for native execution.
More [Comet Memory Tuning](./tuning.md)
diff --git a/searchindex.js b/searchindex.js
index c96b6a4f..63c605b8 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2.
Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark
SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]],
"API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding
Spark-side Tests for the New Expression": [[0,
"adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression":
[[0, [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2.
Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark
SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]],
"API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding
Spark-side Tests for the New Expression": [[0,
"adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression":
[[0, [...]
\ No newline at end of file
diff --git a/user-guide/installation.html b/user-guide/installation.html
index 3b3fb241..32b5475e 100644
--- a/user-guide/installation.html
+++ b/user-guide/installation.html
@@ -382,7 +382,7 @@ under the License.
<section id="requirements">
<h2>Requirements<a class="headerlink" href="#requirements" title="Link to this
heading">¶</a></h2>
<ul class="simple">
-<li><p>Apache Spark 3.3, or 3.4</p></li>
+<li><p>Apache Spark 3.3, 3.4, or 3.5</p></li>
<li><p>JDK 8 and up</p></li>
<li><p>GLIBC 2.17 (Centos 7) and up</p></li>
</ul>
@@ -424,7 +424,7 @@ use of the command <code class="docutils literal
notranslate"><span class="pre">
<section id="run-spark-shell-with-comet-enabled">
<h2>Run Spark Shell with Comet enabled<a class="headerlink"
href="#run-spark-shell-with-comet-enabled" title="Link to this
heading">¶</a></h2>
<p>Make sure <code class="docutils literal notranslate"><span
class="pre">SPARK_HOME</span></code> points to the same Spark version as Comet
was built for.</p>
-<div class="highlight-console notranslate"><div
class="highlight"><pre><span></span><span class="go">export
COMET_JAR=spark/target/comet-spark-spark3.4_2.12-0.1.0-SNAPSHOT.jar</span>
+<div class="highlight-console notranslate"><div
class="highlight"><pre><span></span><span class="go">export
COMET_JAR=spark/target/comet-spark-spark3.4_2.12-0.2.0-SNAPSHOT.jar</span>
<span class="gp">$</span>SPARK_HOME/bin/spark-shell<span class="w">
</span><span class="se">\</span>
<span class="w"> </span>--jars<span class="w"> </span><span
class="nv">$COMET_JAR</span><span class="w"> </span><span class="se">\</span>
@@ -433,7 +433,9 @@ use of the command <code class="docutils literal
notranslate"><span class="pre">
<span class="w"> </span>--conf<span class="w"> </span>spark.plugins<span
class="o">=</span>org.apache.spark.CometPlugin<span class="w"> </span><span
class="se">\</span>
<span class="w"> </span>--conf<span class="w">
</span>spark.comet.enabled<span class="o">=</span><span
class="nb">true</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--conf<span class="w">
</span>spark.comet.exec.enabled<span class="o">=</span><span
class="nb">true</span><span class="w"> </span><span class="se">\</span>
-<span class="w"> </span>--conf<span class="w">
</span>spark.comet.explainFallback.enabled<span class="o">=</span><span
class="nb">true</span>
+<span class="w"> </span>--conf<span class="w">
</span>spark.comet.explainFallback.enabled<span class="o">=</span><span
class="nb">true</span><span class="w"> </span><span class="se">\</span>
+<span class="w"> </span>--conf<span class="w">
</span>spark.driver.memory<span class="o">=</span>1g<span class="w">
</span><span class="se">\</span>
+<span class="w"> </span>--conf<span class="w">
</span>spark.executor.memory<span class="o">=</span>1g
</pre></div>
</div>
<section id="verify-comet-enabled-for-spark-sql-query">
@@ -481,7 +483,7 @@ Comet native shuffle doesn’t support complex types yet.</p>
Spark application. Depending on your deployment mode you may also need to set
the driver & executor class path(s) to
explicitly contain Comet otherwise Spark may use a different class-loader for
the Comet components than its internal
components which will then fail at runtime. For example:</p>
-<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="o">--</span><span
class="n">driver</span><span class="o">-</span><span
class="n">class</span><span class="o">-</span><span class="n">path</span> <span
class="n">spark</span><span class="o">/</span><span
class="n">target</span><span class="o">/</span><span
class="n">comet</span><span class="o">-</span><span class="n">spark</span><span
class="o">-</span><span class="n">spark3</span><span class= [...]
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="o">--</span><span
class="n">driver</span><span class="o">-</span><span
class="n">class</span><span class="o">-</span><span class="n">path</span> <span
class="n">spark</span><span class="o">/</span><span
class="n">target</span><span class="o">/</span><span
class="n">comet</span><span class="o">-</span><span class="n">spark</span><span
class="o">-</span><span class="n">spark3</span><span class= [...]
</pre></div>
</div>
<p>Some cluster managers may require additional configuration, see <a
class="reference external"
href="https://spark.apache.org/docs/latest/cluster-overview.html";>https://spark.apache.org/docs/latest/cluster-overview.html</a></p>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
