This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 9086cfe05 [KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and
2.13
9086cfe05 is described below
commit 9086cfe054969320bc5a29ffbbdd11e2412abfff
Author: PorterZhang2021 <[email protected]>
AuthorDate: Mon Apr 15 09:44:44 2024 +0800
[KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and 2.13
# :mag: Description
## Issue References ๐
This pull request fixes #6297
## Describe Your Solution ๐ง
Here are my main modifications:
1. In the `build/dist` file, I added an additional packaging command that
detects the version of `SCALA_VERSION`. If it is version 2.12, then it will
additionally package version 2.13, otherwise it will package version 2.12
2. In the section of `# Copy spark engines`, the Jar packages of versions
2.12 and 2.13 will be moved to `$DISTDIR/externals/engines/spark/`
This is my first time submitting a PR, so there may be many unfamiliar
areas, and I don't know if my changes meet the needs of the community. More
guidance may be needed. When testing locally, it was able to be packaged
successfully
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6305 from PorterZhang2021/issue-6297.
Closes #6297
efc864373 [PorterZhang2021] Improved task "Package Spark SQL engine both
Scala 2.12 and 2.13" #6297
cb9310bee [PorterZhang2021] Improved task "Package Spark SQL engine both
Scala 2.12 and 2.13" #6297
b6c907509 [PorterZhang2021] Finished task "Package Spark SQL engine both
Scala 2.12 and 2.13" #6297
Authored-by: PorterZhang2021 <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
build/dist | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/build/dist b/build/dist
index 74c04679c..ea58f7a29 100755
--- a/build/dist
+++ b/build/dist
@@ -248,7 +248,18 @@ echo -e "\nBuilding with..."
# shellcheck disable=SC2145
echo -e "\$ ${BUILD_COMMAND[@]}\n"
+# shellcheck disable=SC2050
+if [ "$SCALA_VERSION" = "2.12" ]; then
+ EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@
-Pscala-2.13 -pl :kyuubi-spark-sql-engine_2.13 -am)
+else
+ EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@
-Pscala-2.12 -pl :kyuubi-spark-sql-engine_2.12 -am)
+fi
+
+# shellcheck disable=SC2145
+echo -e "\$ ${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}\n"
+
"${BUILD_COMMAND[@]}"
+"${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}"
# Make directories
rm -rf "$DISTDIR"
@@ -296,7 +307,9 @@ done
cp
"$KYUUBI_HOME/externals/kyuubi-flink-sql-engine/target/kyuubi-flink-sql-engine_${SCALA_VERSION}-${VERSION}.jar"
"$DISTDIR/externals/engines/flink/"
# Copy spark engines
-cp
"$KYUUBI_HOME/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_${SCALA_VERSION}-${VERSION}.jar"
"$DISTDIR/externals/engines/spark/"
+for scala_version in 2.12 2.13; do
+ cp
"$KYUUBI_HOME/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_${scala_version}-${VERSION}.jar"
"$DISTDIR/externals/engines/spark/"
+done
# Copy trino engines
cp
"$KYUUBI_HOME/externals/kyuubi-trino-engine/target/kyuubi-trino-engine_${SCALA_VERSION}-${VERSION}.jar"
"$DISTDIR/externals/engines/trino/"
