This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new bc394a933 [KYUUBI #6305][FOLLOWUP] Improve package Spark SQL engine
both Scala 2.12 and 2.13
bc394a933 is described below
commit bc394a933bfa2f3c5b631476c0e7ee0d9a7f1d83
Author: PorterZhang2021 <[email protected]>
AuthorDate: Wed May 15 12:43:09 2024 +0800
[KYUUBI #6305][FOLLOWUP] Improve package Spark SQL engine both Scala 2.12
and 2.13
# :mag: Description
## Issue References π
This pull request fixes #6305
## Describe Your Solution π§
### Solution 1 use `<profile>` - Inappropriate
I found a way to use <profiles>, roughly as follows:
```xml
<profile>
<id>scala-2.12</id>
<properties>
<scala.binary.version>2.12</scala.binary.version>
</properties>
</profile>
```
After specifying, I attempted to use
1. `build/mvn install -DskipTests -Dmaven.javadoc.skip=true
-Dmaven.scaladoc.skip=true - Dmaven.source.skip -Pscala-2.12 -Pscala-2.13 -pl:
kyuubi-spark-sql-engine -am`
2. `build/mvn install -DskipTests -Dmaven.javadoc.skip=true
-Dmaven.scaladoc.skip=true - Dmaven.source.skip -Pscala-2.13 -Pscala-2.12 -pl:
kyuubi-spark-sql-engine -am`
#### Problem
But in the end, it was found that if both '-Pscala-2.12' and '-Pscala-2.13'
are used at the same time, '-Pscala2.13' will be selected by default, which may
not be a good solution to this problem.
### Solution2
Later, I thought about whether it was possible to filter the parameter '$'
internally. It is effective.
## Types of changes :bookmark:
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan π§ͺ
1. scala2.12
2. scala2.13
---
# Checklist π
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6330 from PorterZhang2021/followup-6297.
Closes #6305
5524b13f5 [Cheng Pan] nit
cccb94d98 [PorterZhang2021] [# 6297] imporve Package Spark SQL engine both
Scala 2.12 and 2.13 Follow Up
fb03d60c2 [Porter Zhang] Merge branch 'apache:master' into followup-6297
393435ea9 [PorterZhang2021] [# 6297] imporve Package Spark SQL engine both
Scala 2.12 and 2.13 Follow Up
0b49f60e6 [Porter Zhang] Merge branch 'apache:master' into followup-6297
f7c7a65c8 [PorterZhang2021] [# 6297] Package Spark SQL engine both Scala
2.12 and 2.13 Follow Up
3d2926afc [Porter Zhang] Merge branch 'apache:master' into followup-6297
eb6406148 [PorterZhang2021] [followup-issue6297] improve issue6297
956ac4955 [PorterZhang2021] [# 6297] Package Spark SQL engine both Scala
2.12 and 2.13 Follow Up
Lead-authored-by: PorterZhang2021 <[email protected]>
Co-authored-by: Porter Zhang
<[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Co-authored-by: PorterZhang2021 <[email protected]>
Co-authored-by: Porter Zhang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
build/dist | 17 ++++++++++++++---
pom.xml | 7 +++++++
2 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/build/dist b/build/dist
index ea58f7a29..b2b4b4abe 100755
--- a/build/dist
+++ b/build/dist
@@ -247,18 +247,28 @@ BUILD_COMMAND=("$MVN" clean install $MVN_DIST_OPT $@)
echo -e "\nBuilding with..."
# shellcheck disable=SC2145
echo -e "\$ ${BUILD_COMMAND[@]}\n"
+"${BUILD_COMMAND[@]}"
+
+FILTERED_ARGS=()
+# shellcheck disable=SC2045
+for arg in "$@"; do
+ if [[ $arg == *scala-2.12* ]]; then
+ FILTERED_ARGS+=("${arg//scala-2.12/scala-2.13}")
+ elif [[ $arg == *scala-2.13* ]]; then
+ FILTERED_ARGS+=("${arg//scala-2.13/scala-2.12}")
+ fi
+done
# shellcheck disable=SC2050
if [ "$SCALA_VERSION" = "2.12" ]; then
- EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@
-Pscala-2.13 -pl :kyuubi-spark-sql-engine_2.13 -am)
+ EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT
${FILTERED_ARGS[@]} -Pscala-2.13 -pl :kyuubi-spark-sql-engine_2.13 -am)
else
- EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@
-Pscala-2.12 -pl :kyuubi-spark-sql-engine_2.12 -am)
+ EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT
${FILTERED_ARGS[@]} -pl :kyuubi-spark-sql-engine_2.12 -am)
fi
# shellcheck disable=SC2145
echo -e "\$ ${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}\n"
-"${BUILD_COMMAND[@]}"
"${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}"
# Make directories
@@ -307,6 +317,7 @@ done
cp
"$KYUUBI_HOME/externals/kyuubi-flink-sql-engine/target/kyuubi-flink-sql-engine_${SCALA_VERSION}-${VERSION}.jar"
"$DISTDIR/externals/engines/flink/"
# Copy spark engines
+# shellcheck disable=SC2045
for scala_version in 2.12 2.13; do
cp
"$KYUUBI_HOME/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_${scala_version}-${VERSION}.jar"
"$DISTDIR/externals/engines/spark/"
done
diff --git a/pom.xml b/pom.xml
index bd29b5427..8773caf0f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1950,6 +1950,13 @@
</properties>
</profile>
+ <profile>
+ <id>scala-2.12</id>
+ <properties>
+ <scala.binary.version>2.12</scala.binary.version>
+ </properties>
+ </profile>
+
<!-- For development only, not generally applicable for all modules -->
<profile>
<id>scala-2.13</id>
