This is an automated email from the ASF dual-hosted git repository.
sunchao pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 5d9d95e feat: Enable Comet shuffle manager for Comet shell (#204)
5d9d95e is described below
commit 5d9d95e9832de516f4fa677e9d6e7cd0134d1bec
Author: Junfan Zhang <junfan.zh...@outlook.com>
AuthorDate: Wed Mar 13 15:03:09 2024 +0800
feat: Enable Comet shuffle manager for Comet shell (#204)
---
bin/comet-spark-shell | 2 ++
1 file changed, 2 insertions(+)
diff --git a/bin/comet-spark-shell b/bin/comet-spark-shell
index 9ae55a1..72100d2 100755
--- a/bin/comet-spark-shell
+++ b/bin/comet-spark-shell
@@ -81,4 +81,6 @@ RUST_BACKTRACE=1 $SPARK_HOME/bin/spark-shell \
--conf spark.comet.enabled=true \
--conf spark.comet.exec.enabled=true \
--conf spark.comet.exec.all.enabled=true \
+ --conf spark.comet.exec.shuffle.enabled=true \
+ --conf
spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager
\
$@
