This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 5e05626c Update installation instructions for spark-shell (#435)
5e05626c is described below
commit 5e05626c970b7ca4d2563b2d97442d3a69256880
Author: Andy Grove <[email protected]>
AuthorDate: Wed May 15 13:22:05 2024 -0600
Update installation instructions for spark-shell (#435)
---
docs/source/user-guide/installation.md | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/docs/source/user-guide/installation.md
b/docs/source/user-guide/installation.md
index efc58f1e..03ecc53e 100644
--- a/docs/source/user-guide/installation.md
+++ b/docs/source/user-guide/installation.md
@@ -57,13 +57,17 @@ Note that the project builds for Scala 2.12 by default but
can be built for Scal
make release PROFILES="-Pspark-3.4 -Pscala-2.13"
```
-## Run Spark with Comet enabled
+## Run Spark Shell with Comet enabled
Make sure `SPARK_HOME` points to the same Spark version as Comet was built for.
```console
+export COMET_JAR=spark/target/comet-spark-spark3.4_2.12-0.1.0-SNAPSHOT.jar
+
$SPARK_HOME/bin/spark-shell \
- --jars spark/target/comet-spark-spark3.4_2.12-0.1.0-SNAPSHOT.jar \
+ --jars $COMET_JAR \
+ --conf spark.driver.extraClassPath=$COMET_JAR \
+ --conf spark.executor.extraClassPath=$COMET_JAR \
--conf spark.sql.extensions=org.apache.comet.CometSparkSessionExtensions \
--conf spark.comet.enabled=true \
--conf spark.comet.exec.enabled=true \
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
